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Section  1.0  INTRODUCTION 


The  availability  and  performance  of 
modern  weapon  systems,  including  ground 
support  systems,  depend  critically  on 
the  subsystems  which  operate  under  the 
control  of  software.  Ground  system  per¬ 
formance,  in  turn,  hinges  on  how  well 
the  functional  and  design  requirements 
for  hardware  and  software  have  been  spec¬ 
ified.  These  requirements  are  the  result 
of  a  derivation  process  encompassing  the 
discipline  of  both  weapon  system  engi¬ 
neering  and  computational  system  engi¬ 
neering.  This  process  of  requirements 
derivation  -  especially  software  require¬ 
ments  -  is  the  principal  topic  of  this 
guidebook.  It  is  described  in  terms  of 
analyses  and  studies  that  are  performed 
and  how  these  relate  to  system  develop¬ 
ment  phasing.  The  particular  systems 
with  which  this  guidebook  is  concerned 
are  automatic  test  equipment  and  train¬ 
ing  simulators. 

1.1  PURPOSE 

The  primary  purpose  of  this  guidebook  is 
to  assist  AF  engineering  personnel  di¬ 
rectly  responsible  for  Training  Simula¬ 
tors  (TS)  and  Automatic  Test  Equipment 
(ATE)  software  acquisition  to  assure  the 
performance  requirements  for  this  soft¬ 
ware  are  successfully  monitored  and 
developed.  The  guidebook  should  also  be 
helpful  to  Air  Force  managers  respon¬ 
sible  for  the  procurement  of  the  total 
TS  or  ATE  systems. 

1.2  SCOPE 

This  is  one  of  a  series  of  guidebooks 
related  to  the  Software  Acquisition  Engir 
neering  (SAE)  process  for  TS  and  ATE 
ground-based  systems.  Other  SAE  guide¬ 
book  titles  are  listed  below: 

Software  Cost  Measuring  and  Reporting 

Requirements  Specification 


Contracting  for  Software  Acquisition 
Statement  of  Work  (SOW)  and  Requests 
for  Proposal  (RFP) 

Regulations,  Specification  and  Stan¬ 
dards 

Measuring  and  Reporting  Software 
Status 

Computer  Program  Documentation  Re¬ 
quirements 

Software  Quality  Assurance 
Verification 

Validation  and  Certification 
Computer  Program  Maintenance 
Software  Configuration  Management 
Reviews  and  Audits 
Management  Reporting 

For  the  purposes  of  this  guidebook,  TS 
requirements  specification  may  be  de¬ 
fined  as  the  process  which  starts  with 
the  gleaning  of  requirements  from  a 
basic  statement  of  need,  such  as  in  a 
Required  Operational  Capabilities  (ROC) 
document  issued  by  a  using  AF  echelon 
and  ends  with  the  collection  of  the 
approved  requirements  in  a  development 
(Part  I)  specification.  This  process  is 
managed  by  the  Air  Force  but  involves 
participation  by  other  organizations; 
e.g.,  weapon  system  prime  contractor  and 
ground  system  suppliers. 

For  ATE,  the  process  normally  starts 
with  an  involved  set  of  maintenance  and 
repair  analyses  performed  by  the  mission 
system  prime  contractor.  This  identifies 
the  support  equipment  which  will  be  re¬ 
quired  for  mission  support  and  also  dif¬ 
ferentiates  between  normal  and  automatic 


equipment.  The  ATE  development  (Part  I) 
specifications  are  written  by  the  mis¬ 
sion  system  prime  contractor  who  can 
either  develop  or  procure  the  ATE. 

1.3  TS  AND  ATE  OVERVIEW 

The  purpose  of  this  section  is  to  pro¬ 
vide  a  brief  sketch  of  TS  and  ATE  system 
characteristics,  including  the  function 
of  the  software  associated  with  each. 

1.3.1  TS  System  Characteristics 

The  TS  system  is  a  combination  of  spe¬ 
cialized  hardware,  computing  equipment, 
and  software  designed  to  provide  a  syn¬ 
thetic  flight  and/or  tactics  environment 
in  which  aircrews  learn,  develop  and 
improve  the  techniques  associated  with 
their  individual  tasks  in  a  specific 
type  aircraft.  In  many  cases,  visual, 
aural,  and/or  motion  systems  may  be  in¬ 
cluded.  Figure  1.3-1  depicts  a  typical 
training  simulator  which  employs  digital 
processing  capability. 

The  computer  system,  integral  to  the 
crew  training  simulator,  consists  of  one 
or  more  general  purpose  computers.  The 
computing  hardware  consists  of  machines 
with  hardware  floating  point  arithmetic 
and  sufficient  bit  capacity  to  provide 
efficient  use  of  the  simulator  High 
Order  Language  (HOL). 

When  a  multi-processor/multi-computer 
system  is  used,  it  must  be  designed  such 
that  all  computers  operate  in  parallel 
in  real-time  and  are  controlled  and  time 
synchronized  from  a  single  computer  pro¬ 
gram  supervisor/executive.  The  executive 
directs  the  program  execution  and  estab¬ 
lished  priorities. 

The  simulator  accepts  control  inputs 
from  the  trainee  via  cockpit  controls 
(or  other  crew  station  controls)  or  from 
the  instructor  operator  station,  per¬ 
forms  a  real-time  solution  of  the  simu¬ 
lator  mathematical  model,  and  provides 
outputs  necessary  to  accurately  repres¬ 
ent  the  static  and  dynamic  behavior  of 
the  real  world  system  within  specified 
tolerance  and  performance  criteria. 


Since  training  simulators  are  a  combina¬ 
tion  of  interdependent  hardware  and  soft¬ 
ware,  a  joint  development  effort  is 
required.  As  the  complexity  of  training 
simulators  increases,  simulation  soft¬ 
ware  continues  to  grow  in  complexity, 
size,  and  cost.  Software  costs  can  and 
do  exceed  computer  hardware  costs  in 
many  cases.  Therefore,  it  is  imperative 
that  the  simulation  software  acquisition 
engineering  process  be  subjected  to  for¬ 
mal  system  engineering  planning  and  dis¬ 
cipline  to  ensure  effective  and  effi¬ 
cient  simulator  procurement. 

1.3.2  ATE  System  Characteristics 

ATE  is  defined  as  that  equipment  which 
is  used  for  maintenance  activities  - 
principally  in  support  of  large  deployed 
systems.  ATE  is  used  in  place  of  manual 
devices  either  because  it  is  more  cost 
effective  or  the  item  being  tested  re¬ 
quires  the  speed  and  timing  which  only 
an  automatic  tester  can  achieve. 

Figure  1.3-2  shows  the  typical  compo¬ 
nents  of  an  ATE  system.  Note  that  there 
are  both  hardware  and  software  elements 
involved.  Most  of  the  elements  shown 
will  be  found  in  one  form  or  another  in 
the  majority  of  ATE  systems. 

The  controls  and  displays  section  con¬ 
sists  of  the  computer  and  peripheral 
devices  like  control  panels,  magnetic 
tape  cassettes  or  disks,  a  cathode  ray 
tube  (CRT)  and  keyboard,  and  usually  a 
small  printer.  The  computer,  as  con¬ 
trolled  by  software,  performs  tasks  like 
operating  the  peripheral  devices,  switch¬ 
ing  test  stimuli  on  and  off,  and  measur¬ 
ing  and  comparing  responses  of  the  Unit 
Under  Test  (UUT)  to  predetermi ned 
values.  The  operator  will  maintain  ulti¬ 
mate  control  of  the  testing  process 
through  some  of  the  peripherals.  How¬ 
ever,  his  interaction  is  usually  minimal 
since,  by  definition,  the  automatic  test 
feature  was  selected  in  preference  to  an 
operator-controlled  test  system.  It  is 
normally  designed  to  allow  a  single  con¬ 
figuration  of  ATE  to  be  used  for  testing 
several  articles  of  system  equipment. 
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ical  Craw  Training  Simulator 


The  maintenance  level  being  supported  by 
the  ATE  Is  determined  by  logistics  sys¬ 
tems  analysis. 

The  importance  of  the  software  portion 
of  the  ATE  system  should  not  be  mini¬ 
mized  since  both  the  application  of  the 
test  stimuli  and  the  measurement  of  the 
result  are  achieved  via  software.  Arbi¬ 
trary  function  generation  and  compli¬ 
cated  wave  analysis  can  also  be  accomp¬ 
lished  by  software. 

1.4  GUIDEBOOK  ORGANIZATION 

The  scope  and  purpose  of  this  guidebook, 
as  well  as  the  general  characteristics 
of  TS  and  ATE  systems,  are  defined  in 
the  Introduction  (Section  1.0). 

Software  requirements  specification  for 
TS  and  ATE  is  discussed  in  two  separate 
sections:  Sections  3.0  and  4.0,  respec¬ 
tively.  Each  of  these  two  sections  is 
subdivided  according  to  major  activ¬ 
ities/milestones  in  the  process  of 


deriving  specifications.  This  process 
begins  with  requirements  for  the  weapon 
system  being  supported  by  TS  or  ATE  and 
ends  with  the  procurement  specifications 
for  hardware  and  software.  The  process 
is  specific  to  each  type  of  ground  sys¬ 
tem  and  is  described  in  the  Introduction 
to  each  section. 

Documents  which  are  most  directly  appli¬ 
cable  to  the  subject  of  TS  or  ATE 
requirements  specification  are  listed  in 
Section  2.0.  Additional  supporting  docu¬ 
mentation  is  identified  in  the  Biblio¬ 
graphy,  Section  5.0. 

The  relationship  of  guidebook  topics  to 
specific  paragraphs  in  government  docu¬ 
ments  is  described  by  a  matrix  format  in 
Section  6.0.  A  detailed  subject  index  to 
the  guidebook  is  provided  in  Section 
9.0. 

A  glossary  of  terms  and  abbreviations/ 
acronyms  are  provided  in  Sections  7.0 
and  8.0,  respectively. 
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Section  2.0  APPLICABLE  DOCUMENTS 


The  following  documents  bear  directly  on 
the  topic  of  requirements  specification 
for  ATE  and  TS  software: 

DOD  5000.29,  Management  of  Computer 
Resources  in  Major  Defense  Systems, 
26  April  1976 

AFR  800-14  Vol.  II,  Acquisition  and 
Support  Procedures  for  Computer  Re¬ 
sources  in  Systems,  26  September  1975 

MIL-STD-483,  Configuration  Management 
Practices  for  Systems,  Equipment,  Mun¬ 
itions,  and  Computer  Programs,  1  June 
1971 

MIL-STD-490,  Military  Standard  Speci¬ 
fication  Practices,  18  May  1972 

MIL-STD-1519,  Preparation  of  Test 
Requirements  Documentation,  17  Septem¬ 
ber  1971 


MIL-D-83468,  Digital  Computational 
System  for  Real-Time  Training  Simu¬ 
lators,  12  December  1975 

AFLC  Regulation  66-37,  Management  of 
Automated  Test  Systems,  24  October 
1975 

MIL-S-83490,  Specifications,  Types 
and  Forms,  30  October  1968 

MIL-STD-499A,  Engineering  Management, 
1  May  1974 

AFR  57-1,  Required  Operational  Capa¬ 
bilities,  30  May  1975 

AFM  50-2,  Instructional  System  Devel¬ 
opment 

AFP  50-58,  Handbook  for  Designers  of 
Instructional  Systems,  Vol.  1-5 

AFR  800-2,  Outline  1 
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Section  3.0  TRAINER  SIMULATOR  SOFTWARE  REQUIREMENTS  SPECIFICATION 


This  section  describes  the  software 
acquisition  engineering  (SAE)  process 
necessary  to  derive  software  require¬ 
ments  for  a  TS  system.  This  derivation 
begins  with  a  ROC  and  concludes  with 
documentation  of  specific  software  re¬ 
quirements.  The  SAE  process  involves 
three  principal  tasks: 

a.  Technical  evaluation  (process  of 
deriving  software  requirements) 

b.  Planning  (definition  of  TS  develop¬ 
ment  approach) 

c.  Documentation  (description/specifi¬ 
cation  of  req  ui  rente  nts) 

Section  3.0  is  organized  under  these 
principal  tasks. 

The  preparation  and  issuance  of  a  ROC 
defines  a  need  for  services  and/or  equip¬ 
ment  (hardware/software)  to  provide  a 
specific  TS  capability.  A  ROC  defining 
training  simulation  needs  concerns  the 
training  of  personnel  to  operate  or  main¬ 
tain  a  mission  vehicle  and  related  equip¬ 
ment.  An  orderly  process  is  followed  for 
planning  and  developing  an  instructional 
program  which  insures  that  crew  person¬ 
nel  are  taught  the  knowledge,  skills, 
and  attitudes  essential  for  successful 
job  performance.  Requirements  for  a  TS  - 
both  hardware  and  software  -  are  speci¬ 
fied  in  the  .ontext  of  facilitating  that 
i ns truct i onal  program. 

The  Air  Force  TS  software  engineer  is 
involved  in  the  process  of  requirements 
specification  from  initial  Air  Force 
Systems  Command  (AFSC)  review  of  a  ROC 
through  the  last  negotiated  change  to  TS 
requirements  (which  can  occur  long  after 
the  TS  procurement  specification  is 
finalized).  Emphasis  in  this  guidebook 
is  placed  on  those  activities  up  to,  and 
including,  final  approval  of  the  TS  re¬ 
quirements  specification. 


The  principal  task  of  the  AF  TS  software 
engineer  in  requirements  specification 
is  to  interpret  and  augment  MIL-D-83468 
for  the  specific  TS  system  being  devel¬ 
oped.  This  means  tailoring  MIL-D-83468, 
item  by  item,  to  match  the  particular 
objectives  and  unique  features  of  the 
proposed  TS.  A  MIL-D-83468  checklist  is 
provided  in  paragraph  3.1  to  assist  this 
activity. 


It  is  important  to  note  at  the  outset 
that  TS  software  requirements  cannot  be 
derived  independently  from  TS  hardware. 
TS  software  and  hardware  are  interdepen¬ 
dent  and  further,  the  implementation  of 
the  TS  functional  requirements  can  con¬ 
sider  trades  between  hardware  and  soft¬ 
ware  capabilities. 

Since  TS  requirements  are  derived  for  an 
integrated  hardware/software  system,  the 
AF  TS  software  engineer  will  participate 
in  requirements  derivation  as  a  team  mem¬ 
ber.  This  team,  the  System  Program  Of¬ 
fice  (SPO)  cadre  and  associated  consul¬ 
tants,  will  develop  and  select  a  TS 
design  concept  which  meets  user  require¬ 
ments  at  acceptable  cost  and  risk. 

System  selection  under  these  criteria 
often  involves  the  use  of  off-the-shelf 
hardware/software  components  -  another 
reason  software  requirements  cannot  be 
divorced  form  integrated  TS  system 
requirements. 

Mary  major  manufacturers  of  training  sys¬ 
tems  have  developed  standard  modules  and 
high  technology  software  which  facili¬ 
tate  their  ability  to  provide  simulator 
systems  meeting  a  wide  variety  of  re¬ 
quirements.  This  is  accomplished  by 
assembling  (and  providing  modifications 
to)  a  number  of  standard  hardware  and 
software  modules  tailored  to  a  specific 
TS  capability.  Consequently  the  manufac¬ 
turer  will  have  already  made  hardware/ 
software  trades  for  these  modules  and 
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selected  an  approach  which  enhances  his 
ability  to  remain  competitive,  both  in 
cost  and  performance.  If  the  TS  acquisi¬ 
tion  engineer  places  quantitative  soft¬ 
ware  performance  requirements  in  the  pro¬ 
curement  specification,  these  require¬ 
ments  may  negate  the  contractor's  own 
efforts  to  achieve  cost  effectiveness; 
with  the  result  that  the  Increased  cost 
and  associated  technical  risk  necessary 
to  meet  these  quantitative  requirements 
is  passed  on  to  the  government.  In  gen¬ 
eral,  quantitative  performance  require¬ 
ments  should  be  specified  at  the  system 
level,  leaving  to  the  contractor  such 
decisions  as  whether  a  performance 
requirement  is  met  by  hardware,  soft¬ 
ware,  or  a  combination  of  these. 

Hence,  the  software  requirements  deter¬ 
mination  and  specification  should  not  be 
divorced  from  system  and  hardware  consid¬ 
eration  and  the  software  acquisition 
engineer  is  a  key  part  of  the  SPO  cadre. 
Further,  specific  software  requirements 
contained  in  the  RFP  for  TS  should 
generally  be  limited  to  qualitative 
requirements  of  the  type  contained  in 
MIL-D-83468.  Once  the  contractor  has 
interpreted  the  TS  requirements  in  his 
proposal  response  to  the  RFP,  then  more 
specific  software  requirements  can  be 
included  in  the  final  procurement  speci¬ 
fication  which  is  negotiated. 

The  process  of  developing  analysis  and 
data  for  input  to  the  TS  specification 
is  described  in  paragraph  3.1.  This 
process  is  explained  by  the  sequence  of 
major  events,  description  of  specifica¬ 
tion  activities,  and  relationship  of 
supporting  documentation. 

The  contents  of  planning  documentation, 
which  supports  the  requirements  deriva¬ 
tion  process,  are  described  in  paragraph 

3.2.  Then,  actual  preparation  of  the  TS 
specification  is  discussed  in  paragraph 

3.3.  This  discussion  of  specification 
preparation  relates  how  the  requirements 
derivation  data/analyses  (paragraph  3.1) 
and  planning  elements  (paragraph  3.2) 
are  used  to  produce  the  TS  specification. 


Two  additional  paragraphs  in  Section  3.0 
are  Problem  Areas  (paragraph  3.4)  and 
Conclusions  (paragraph  3.5). 

The  flight  crew  TS  is  used  as  the  prin¬ 
cipal  example  in  this  guidebook.  How¬ 
ever,  this  process  of  deriving  software 
requirements  is  generally  applicable  to 
other  TS  (i.e. ,  for  other  mission  sta¬ 
tions  and  maintenance  positions). 

3.1  TECHNICAL  EVALUATION 

The  term  "technical  evaluation"  is  used 
in  this  guidebook  to  describe  the  pro¬ 
cess  for  developing  software  require¬ 
ments  for  TS  systems.  Figure  3.1-1  Illus¬ 
trates,  in  general,  how  TS  software 
requirements  evolve  from  a  ROC.  The 
progression  is  from  the  ROC  to  the  TS 
system,  from  the  TS  system  to  the  compu¬ 
tational  system  and  the  allocation  of 
requirements  to  hardware  and  software 
within  the  computation  system.  In  actual 
practice,  the  flow  is  not  always  direct. 
There  are  iterative  paths  and  interdepen¬ 
dencies  between  "levels"  of  requirement 
specification. 

The  goal  of  technical  evaluation  is  to 
develop  supporting  data  for  TS  require¬ 
ments  that  are  technically  feasible,  res¬ 
ponsive  to  ROC  requirements  and  within 
cost  constraints.  These  supporting  data 
are  then  utilized  for  preparation  of  TS 
procurement  specifications  (paragraph 
3.3). 

The  process  of  TS  software  requirements 
derivation  is  described  in  two  principal 
ways: 

a.  The  sequence  of  major  events  (mile¬ 
stones)  leading  up  to  final  approval  of 
the  TS  requirements  specification,  and 

b.  The  major  derivation  activities 
associated  with  those  events. 

This  description  of  the  process  is  organ¬ 
ized  in  this  section  under  the  major  der¬ 
ivation  activities  -  as  shown  in  Figure 
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3.1-2.  Also  shown  in  the  figure  are  orga¬ 
nizations  having  principal  responsibili¬ 
ties  for  the  identified  activities.  Each 
activity  is  described  in  the  Indicated 
section  paragraphs. 

The  sequence  of  major  events  in  TS  re¬ 
quirements  derivation  is  Illustrated  in 
Figure  3.1-3.  These  are  then  tied  to 
derivation  activities  in  Figure  3.1-4. 
The  figure  provides  a  good  overview  of 
the  requirements  derivation  process  and 
warrants  careful  inspection.  Arrows 
between  the  boxes  in  the  diagram  signify 
information  exchange,  for  example,  the 
definition  of  candidate  TS  systems 
depends  on  inputs  from  (1)  definition  of 
training  simulation  requirements,  (2)  TS 
systems  analyses  and  trades,  and  (3)  TS 
preliminary  design.  Also,  some  relation¬ 
ships  are  double-arrowed,  for  example 
between  "TS  systems  analysis  and  trades" 
and  "TS  preliminary  design".  This  means 
the  process  is  iterative  and  neither 
activity  is  completed  until  both  are 
completed.  Each  of  the  primary  activi¬ 
ties  is  discussed  in  paragraphs  3.1.1 
through  3.1.6. 

The  relationship  of  supporting  documents 
to  the  process  events  and  activities  is 
shown  in  Figure  3.1-5.  Not  all  relation¬ 
ships  between  process  elements  are  shown 
in  Figure  3.1-5  (additional  lines  and 
arrows  would  clutter  the  figure)  but 
principal  relationships  are  indicated. 
Figure  3.1-5  provides  a  composite  view 
of  elements  in  the  requirements  deriva¬ 
tion  process  and,  although  the  diagram 
is  somewhat  involved,  the  process  is 
rather  straightforward  when  each  activ¬ 
ity/event  is  considered  individually  (in 
the  following  paragraphs). 

As  a  further  aid  to  tracking  the  process 
of  TS  requirements  specification,  a 
detailed  checklist  of  specific  events 
was  developed.  This  checklist  is  given 
in  Table  3.1-1.  Significant  events  in 
the  requirements  specification  process 
are  arranged  in  their  chronological 
order.  The  table  also  has  a  column  to 
record  the  planned  completion  date,  the 
current  status  and  the  date  the  event  is 
completed.  The  table  can  be  used  as  a 


convenient  checklist  for  planning  and 
evaluating  the  requirements  derivation. 
Not  all  TS  requirements  developments 
will  follow  this  exact  sequence  of  tasks 
but  the  checklist  can  be  modified  to 
suit  different  development  approaches. 
This  checklist  is  a  composite  of  two  AF 
procedures.  Items  1  to  5  are  from  AFR 
800-2,  outline  1.  Items  6  to  11  result 
from  a  review  of  AFP  50-58  (Handbook  for 
Designers  of  Instructional  Systems). 

3.1.1  ROC  Review 

The  ROC  is  examined  to  discern  required 
TS  system  characteristics,  mission  objec¬ 
tives  and  functions,  and  minimum  accep¬ 
table  system-level  technical  performance 
requirements.  The  following  are  examples 
of  TS  functions:* 

a.  Simulate  selected  on-board  systems 
operations 

b.  Simulate  weapon.  system  physiologi¬ 
cal  environment 

c.  Simulate  weapon  system  operation 
environment 

d.  Provide  instructor  control  fea¬ 
tures 

e.  Provide  advanced  instructional 
features 

Simulation  is  an  approximation  or  repre¬ 
sentation  of  real  world  phenomena.  A  suc¬ 
cessful  training  simulation  is  one  in 
which  the  student  perceives  "realistic" 
sensory  inputs  and  system  responses;  at 
least  with  sufficient  fidelity  to  pre¬ 
pare  the  student  for  actual  operational 
situations.  Additional  criteria  for  a 
successful  TS  are  that  the  TS  provides 
(1)  the  range  and  diversity  of  situa¬ 
tions  associated  with  crew  personnel 
duties  and  mission  tasks,  (2)  feedback 
to  the  student  as  rewards/penalties  for 
specific  behaviors,  and  (3)  Instructor 
monitoring  and  evaluation  of  trainee 
performance. 

Technical  performance  statements  in  the 
ROC  may  be  stated  either  qualitatively 
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Figure  3. 1-4.  Relationship  of  Derivation  Activities  to  Major  Events 


Figure  3. 1-5.  Composite  of  Requirement  Process  Elements 
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Table  3. 1 •  1.  Requirements  Specification  Event  Checklist  (Sheet  1  of  3) 


EVENT 

1.  USING  COMMAND  SUBMITS  ROC  TO  HQ  USAF 

2.  HQ  USAF  DISTRIBUTES  ROC  TO  USAF  AGENCIES 

3.  AGENCIES  REVIEW  AND  RETURN  ROC  TO  HQ  USAF 

4.  HQ  USAF  APPROVES  ROC  AND  ISSUES  PMD 

5.  ASFC  DIVISION  FORMS  A  SPO  CADRE 

6.  STUDY  EFFORT  TO  DETERMINE  MEANS  TO 
SATISFY  ROC 

a.  STUDY  GROUP  RECEIVES  ROC  AND  PMD 

b.  T.I.  MEETING  TO  DEFINE  STUDY  OBJECTIVES 
AND  PRODUCTS  OF  STUDY 

SPECIFIC  EFFORTS  INCLUDE: 

•  ESTABLISH  TRAINING/SIMULATION 
OBJECTIVE 

•  ESTABLISH  STUDY  SCHEDULE 

•  DETERMINE  JOB  SYSTEM  PERFORMANCE 
REQUIREMENTS 

•  DETERMINE  THE  TRAINING/SIMULATION 
EQUIPMENT  (HW  &  SW)  REQUIREMENTS 

•  REVIEW  SYSTEM  EQUIPMENT 

•  PREPARE  LIST  OF  TRAINING  EQUIPMENT 

(HW  &  SW)  AND  PRELIMINARY  DESIGN  LAYOUT 

•  PREPARE  DEVELOPMENT  CONCEPT  PAPER  (DCP) 

c.  COMPLETE  TASK/FUNCTION  DESCRIPTION  WORKSHEETS 

d.  COMPLETE  CRITERION  OBJECTIVE  AND  TRAINING/ 
SIMULATION  REQUIREMENTS  WORK  SHEETS 

e.  ESTABLISH  TRAINING  REQUIREMENTS 
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Table  3. 1- 1.  Requirements  Specification  Event  Checklist  (Sheet  2  of  3) 


1 

i 


EVENT 

6.  f.  COMPLETE  TRAINING/SIMULATION  MEDIA 

TRADE  STUDY 

g.  COMPLETE  SURVEY  STUDY  OF  TRAINING/ 
SIMULATION  SYSTEMS  AND  METHODS 

h.  T.I.  MEETING 

•  PRESENT  PRELIMINARY  OUTLINE  OF  DCP 

•  FINALIZE  CRITERION  OBJECTIVES  AND 
RECOMMENDED  TYPE  OF  TRAINING/ 
SIMULATION  MEDIA 

•  ESTABLISH  STUDY'S  TRAINING/SIMULATION 
BASELINE  TO  BEGIN  EQUIPMENT  SELECTION 

•  ESTABLISH  GUIDELINE  FOR  SPECIFICATIONS 

i.  COMPLETE  PRELIMINARY  EQUIPMENT  (HW  &  SW) 
SELECTIONS 

j.  COMPLETE  PRELIMINARY  DRAFT  OF  DCP 

7.  SUBMIT  DCP  FOR  REVIEW 

8.  DETERMINE  METHOD  OF  SPECIFICATION 

•  SYSTEM  SPECIFICATION 

•  HARDWARE  SPECIFICATION 

•  SOFTWARE  SPECIFICATION 

9.  DEVELOP  DRAFT  REQUIREMENTS  SPECIFICATIONS 

•  SYSTEM  SPECIFICATION 

•  HARDWARE  SPECIFICATION  (IF  REQUIRED) 

•  SOFTWARE  SPECIFICATION  (IF  REQUIRED) 
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EVENT 

10. 

T.I.  MEETING 

REVIEW  DCP 

REVIEW  DRAFT  SPECIFICATIONS 

DRAFT  SOW 

11. 

SUBMIT  VISIBILITY  SPECIFICATIONS  AND  DCP 

12. 

ISSUE  RFP 

13. 

COMPLETE  SOURCE  SELECTION  (SELECT  CONTRACTOR) 

14. 

SUBMIT  FINAL  REQUIREMENTS  SPECIFICATION 

15. 

PLACE  CONTRACTOR  PROPOSAL  UNDER  CONTRACT 

■ 

or  quantitatively.  For  example,  the  ROC 
may  state  that  the  trainer  is  to  be  like 
those  used  with  a  given  aircraft  by  com¬ 
mercial  airlines.  This  is  a  qualitative 
indication  to  TS  engineers  of  the  scope 
and  nature  of  the  system  the  users  have 
in  mind  for  the  trainer,  even  though  it 
is  not  a  wholly  definitive  one. 

At  this  point  in  the  evaluation,  there 
is  no  attempt  to  differentiate  between 
hardware  and  software  functions  or  sub¬ 
functions,  except  for  those  which  may 
have  been  explicitly  stated  in  the  ROC. 
However,  the  characteristics,  objec¬ 
tives,  etc.,  which  are  included  in  the 
ROC,  need  to  be  examined  for  feasibility 
and  attainability  with  respect  to  cur¬ 
rent  trainer  technology,  physical  re¬ 
sources,  human  (trainer  and  instructor) 
performance  capabilities,  life  cycle 
costs,  and  other  constraints.  The  simula¬ 
tor  engineer(s)  has  signficiant  contri¬ 
bution  to  make  relative  to  this  task, 
based  on  his  experience  with  other  TS 
systems. 

As  noted  in  Figure  3.1-4,  the  ROC  review 
activity  provides  input  to  (1)  defini¬ 
tion  of  training  simulation  requirements 
and  (2)  TS  systems  analyses  and  trades. 
This  activity  is  conducted  by  two  princi¬ 
pal  organizations  (Figure  3.1-2):  an  AF 
Instructional  System  Development  (ISD) 
team  and  the  SPO  cadre.  The  major  task 
of  this  joint  effort  is  to  begin  defin¬ 
ing  training  simulation  requirements. 

Both  the  SPO  personnel  subsystem/train¬ 
ing  equipment  manager,  and  designated 
USAF  agency  training  equipment  manager 
will  be  potential  co-chairman  of  the  SPO 
cadre.  The  cadre  will  usually  consist  of 
the  designated  USAF  agency  coordinator 
and  AF  subject  matter  specialists,  desig¬ 
nated  specialists  in  training  services 
and  equipment,  representatives  of  the 
weapon  system  using  command  and  other  AF 
agencies  as  required.  Weapon  system  and 
TS  contractor  personnel  are  also  key 
participants. 

Procedures  for  analysis  of  training 
objectives  and  requirements  are  those 


inherent  in  the  ISD  process.  The  ISD  pro¬ 
cess  is  described  in  AFM  50-2  and  the 
Handbook  for  Designers  of  Instructional 
Systems,  AFP  50-58. 

3.1.2  TS  Systems  Analysis  and  Trades 

The  technical  evaluation  process  in¬ 
cludes  various  analyses  and  trade-offs 
to  translate  the  overall  system  require¬ 
ments  statements  in  the  ROC  to  a  spe¬ 
cific  set  of  requirements  for  the  TS 
system  (Figure  3.1-4).  Analysis  and 
trade-off  techniques  are  employed  to 
select  a  set  of  requirements  for  a  sys¬ 
tem  that  can  be  produced  within  allow¬ 
able  costs,  has  low  technical  risk,  is 
within  current  state-of-the-art  and  is 
responsive  to  user  needs.  This  activity 
depends  on  inputs  from  the  ROC  review 
and  from  TS  preliminary  design.  As  noted 
in  Figure  3.1-4,  this  activity  is  inter¬ 
active  with  TS  preliminary  design. 

Discussion  of  TS  systems  analysis  and 
trades  is  approached  from  two  direc¬ 
tions: 

a.  How  ROC-derived  requirements  and 
preliminary  design  interact  to  establish 
training  system  requirements,  including 
software  requirements. 

b.  Examples  of  representative  analy¬ 
ses  and  trades. 

These  are  treated  separately  in  the  next 
two  sections. 

3. 1.2.1  Training  System  Requirements 
for  Flight  Simulators.  The  experience  of 
flight  in  training  simulation  can  range 
from  a  minimum  of  Horizontal  Situation 
Indicators  (HSI)  and  Attitude  Direction 
Tndicators  (ADI)  for  the  pilot  to  a  maxi¬ 
mum  of  out-of-the-window  view,  cockpit 
u  *ion,  audio  cues  and  fully  operational 
-kpit  controls. 

The  basic  hardware  components  of  a 
flight  simulator  are  a  computer,  a  cock¬ 
pit  and  an  interface.  Selection  of  re¬ 
finements  such  as  a  motion  base,  a  vis¬ 
ual  display  system,  audio  cues  and  the 
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instrument/control  complement  will  scope 
the  detailed  interface  configuration  and 
specific  software  requirements. 

The  total  system  includes  computer  pe¬ 
ripherals  for  input/output  capabilities 
and  utility  software  to  be  used  in  soft¬ 
ware  development  and  operations. 

Model  requirements  for  a  TS  system  can 
be  derived  from  the  required  sensory 
inputs  to  the  student,  as  specified  or 
interpreted  from  the  ROC.  For  the  case 
of  a  flight  simulator,  the  major  cate¬ 
gories  of  input  are: 

a.  Cockpit  displays 

b.  Visual  display 

c.  Motion 

d.  Audio  cues 

e.  Control  loading 

These  i terns  translate  into  hardware  and 
software  components  and  Table  3.1-2  pro¬ 
vides  an  example  of  such  a  translation. 

Real-time  software  packages  to  support 
flight  simulation  are  flow  charted  in 
Figure  3.1-6.  Usefulness  of  the  simula¬ 
tor  as  a  training  tool  is  facilitated  by 
means  of  instructor- interactive  software 
for  malfunction  insertion/deletion; 
flight  condition  selection;  mode  control 
and  other  features.  The  executive  pro¬ 
gram  (Figure  3.1-6)  working  input/output 
(I/O)  routines  and  interrupt  handlers 
provide  trainer  controllability. 

The  balance  of  the  software  system  is 
made  up  of  non-real -time  processors, 
utilities,  and  diagnostics  which  provide 
training  flexibility  and  maintenance 
capabilities.  Processors  are  assemblers 
and  compilers.  Examples  of  utilities  are 
source  edit  programs,  link-loaders, 
file-merge/delete  routines,  dump  rou¬ 
tines  and  debug  packages.  Diagnostics 
are  programs  which  exercise  hardware, 
usually  by  causing  information  transmit¬ 
tal  through  critical  interfaces.  Exam¬ 
ples  of  diagnostics  are: 


a.  Micro  level  checking  of  computer 
Central  Processing  Unit  (CPU)  capability 
to  executive  instructions 

b.  Checking  memory 

c.  Exercising  I/O  interfaces,  check¬ 
ing  status  indicators,  parity 

d.  Exercising  standard  peripherals, 
checking  peripheral  response  to  control 
and  data 

e.  Exercising  external  interfaces  and 
simulator-unique  hardware,  checking  sig¬ 
nal  returns,  indicators,  and  physical 
displacements 

Simulation  hardware  requirements  are 
both  general  purpose  (a  digital  com¬ 
puter,  its  peripherals,  analog/digital, 
digital /agalog  converters  and  discrete 
lines)  and  special  purpose  (interface 
logic  and  drivers  for  the  cockpit  dis¬ 
play  and  control  inputs).  In  the  same 
way,  software  requirements  exist  for 
off-the-shelf  processors  -  assembler, 
compiler,  utilities  -  and  for  programs 
specially  written  to  model  the  particu¬ 
lar  airplane  subsystems. 

Software  requirement  derivation  and  hard¬ 
ware  requirement  derivation  are  pro¬ 
cesses  that  interact  with  each  other. 

3. 1.2.2  Examples  of  Analyses  and 
Trades.  Following  ROC  evaluation,  a  com¬ 
plete  list  of  the  specific  TS  system 
functions  and  subfunctions  is  derived. 
Some  may  be  in  the  ROC,  but  others  may 
need  to  be  established  by  additional 
coordination  or  analysis.  Representative 
trainer  mission  functions  are  described 
below  alone  with  examples  of  subfunc¬ 
tions  that  could  be  associated  with  each 
one. 

a.  Simulate  Selected  On-Board  Systems 
Operations  -  THe  subfunctions  which  may 
be  included  depend  partly  on  the  type  of 
weapon  system  which  is  involved,  but 
examples  are: 
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Table  3. 1-2  Real  Time  Model  Software  Items 


Hardware 

Required 

Function 

Interface 

Cockpit 

Software 

Required 

Cockpit  displays 

Digital/Analog 
converters. 
Discrete  outputs 
Synchro  outputs 
Discrete  inputs 

Pilots'  and 
Flight  Engineers 
instruments, 
gauges  and 
lights 

Models  for  each  flight 
subsystem;  engines 
hydraulics,  electrics, 
radio  aids 

Visual  display 

Digital/Analog 
converters, 
synchro  outputs, 
discrete  outputs 

Image  acquisi¬ 
tion,  projection 
equipment 

Algorithms  to  produce 
drive  to  image  acquisi¬ 
tion  equipment,  given 
translational,  rotational 
parameters  from  airplane 
dynamics  solution 

Motion 

Digital/Analog 
conv. ,  Analog/ 
digit,  convert, 
discrete  outputs 

Hydraulical ly 
driven  motion 
base  hydraulic 
power  supply 

Algorithms  to  produce 
drives  to  hydraulic- 
control  amplifiers  given 
transl.,  rotational 
description  parameters 
from  airplane  dynamics 
solution 

Audio  cues 

Discrete  outputs, 

digital/analog 

converters 

Sound  synthe¬ 
sizers,  ampli¬ 
fiers,  speakers 

Algorithms  to  drive 
synthesizers  given  status 
of  subsystems  related  to 
sound  sources;  engines, 
hydraulics,  etc. 

Control  loading 

Digital/analog, 

analog/digita! 

converters 

Control -drive 

hydraulics, 

amplifiers 

Data  for  flight 
condition,  load  factor, 
hinge  moments  and 
blowdown  limits. 
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(1)  On-board  weapons  systems  such 
as  air-to-air  and  air-to-ground  mis¬ 
siles,  gravity  weapons,  cruise  mis¬ 
siles  and  rockets 

(2)  Flight  control  system 

(3)  Communications 

(4)  Flight  instruments 

(5)  Navigation 

b.  Provide  Instructor  Control  Features 
-  Instructor  control  is  affected  by  fac- 
tors  such  as  simultaneous  instructor  con¬ 
trol  of  multiple  trainee  positions  and 
the  number  of  instructor  positions. 
Another  factor  is  the  instructor's 
requirement  to  be  able  to  override  or 
reset  processes  initiated  by  trainees. 
In  addition,  there  is  a  host  of  stimuli 
and  conditions  which  the  instructor  may 
have  to  control  at  each  position. 

c.  Provide  Advanced  Instructional  Fea¬ 
tures  -  Several  automatic  features  may 
be  specified: 

(1)  Provide  automatic  control  of 
initial  conditions 

(2)  Provide  automatic  demonstra¬ 
tion 

(3)  Provide  automatic  malfunc¬ 
tion  insertion 

(4)  Provide  automatic  monitoring 
of  procedural  items 

(5)  Provide  automatic  permanent 
recording  of  results 

(6)  Provide  student's  feedback 
capability 

(7)  Provide  automated  perfor¬ 
mance  comparisons. 

Examples  of  three  specific  analyses  are: 

a.  Visual  Display  Tradeoff  -  A  visual 
display  system  is  required.  If  the 


simulator  is  planned  for  night  flight 
training,  could  one  of  the  computer¬ 
generated  CRT  displays  suffice?  Note 
that  in  going  to  a  CRT-type  display,  a 
peripheral  mini  computer  is  probably 
needed  for  refresh  of  the  CRT  image. 
Appropriate  software  to  interface 
between  the  simulator  computer  and  the 
mini  is  needed.  In  trade  for  this  added 
cost  is  relief  from  the  electromechan¬ 
ical  complexity  of  image  generation 
using  a  moving  television  camera. 

b.  Life  Cycle  Cost  (LCC)  Analysis 
The  subject  of  LCC  is  discussed  in  the 
Cost  Measuring  and  Reporting  SAE  Guide¬ 
book.  A  brief  summary  of  this  discipline 
is  provided  herein,  both  as  an  example 
of  an  important  analysis  and  because  LCC 
appears  as  a  parameter  in  other  studies. 
Experience  has  shown  that  early  deci¬ 
sions  in  system  concept  and  definition 
phases  have  the  greatest  potential  for 
cost  savings.  Experience  utilizing  cur¬ 
rent  LCC  financial  reporting  techniques, 
in  particular  the  Air  Force  Logistics 
Command  (AFLC)  model  for  logistic  sup¬ 
port  costs,  enables  the  implementation 
and  utilization  of  an  effective  LCC  pro¬ 
gram  to  assess  TS  software  during  this 
critical  point  in  development.  Cost 
drivers  are  defined,  challenged,  and 
trade  studies  made  to  reduce  the  impact 
of  the  cost  drivers  on  the  support 
costs.  Cost-avoidance  disciplines,  in¬ 
cluding  design  simplification  and  opti¬ 
mum  use  of  standard  modules  should  be 
stressed  early  in  the  formulation  of  TS 
requirements.  The  20-year  LCC  analysis 
also  provides  the  means  for  establishing 
cost  targets,  monitoring  acquisition 
costs,  and  instituting  corrective 
action. 

The  LCC  process  is  summarized  in  Figure 
3.1-7.  A  baseline  system  for  the  TS  is 
initially  established  from  which  trade 
studies  are  conducted  on  various  alter¬ 
natives.  The  baseline  requirements  are 
determined,  assessment  of  the  reliabili¬ 
ty  (R)  and  maintainability  (M)  made,  and 
logistics  support  analysis  (LSA)  of  the 
TS  is  performed,  based  on  MIL-STD-1388. 
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The  Logistics  Support  Costs  (LSC)  model 
Input  data  Includes  parameters  such  as 
repair  cycle  times  and  labor  rates.  This 
data,  together  with  the  Logistics  Sup¬ 
port  Analysis  (LSA)  data  and  the  unit 
costs,  complete  the  LSC  model  data 
requirements. 

The  LCC  of  the  baseline  TS  concept  is 
then  determined  from  the  LSC,  acquisi¬ 
tion  costs  and  development  costs.  The 
LSC  data  is  used  in  trade  studies  to 
identify  cost  drivers  and  candidate 
alternative  approaches  are  measured  in 
the  model.  Full  consideration  is  given 
to  adopting  existing  hardware  and  soft¬ 
ware  for  use  in  reducing  LCC.  Other  sys¬ 
tems  in  being,  or  planned  for  activa¬ 
tion,  are  reviewed  and  approaches  eval¬ 
uated  for  use  of  common  support  require¬ 
ments  to  the  extent  permitted  by  the 
development  concepts.  The  overall  trade 
study  process  is  a  multiple-disciplined 
effort  involving  procurement  engineer¬ 
ing,  test  and  logistics  disciplines. 
After  each  trade  study  has  been  com¬ 
plete,  a  detailed  evaluation  of  the 
results  is  performed  and  changes  to  the 
TS  baseline  evaluated. 

c.  Risk  Management  -  A  key  element  is 
the  requirement  specification  for  the  TS 
system  and  its  risk  assessment. 

As  previously  stated,  a  principal  goal 
of  the  configuration  engineer  should  be 
to  minimize  risk  to  the  maximum  extent 
practical,  consistent  with  supporting 
requirements  reflected  in  the  ROC  and 
its  supporting  documentation.  Such  fac¬ 
tors  as  the  existence  of  "off-the-shelf" 
or  easily-modiflable  software  and  hard¬ 
ware  is  a  significant  factor  affecting 
both  schedule  and  cost.  Figure  3.1-8  and 
Table  3.1-3  illustrates  a  formal  means 
whereby  high  risk  items  are  identified 
and  continually  reviewed  while  risk 
abatement  action  is  taken.  Also  pre¬ 
sented  are  example  criteria  for  judging 
whether  any  particular  risk  item  is  of 
sufficient  magnitude  to  treat  In  this 
manner.  Risk  assessment  is  a  continuing 
process  and  normally  is  reported  at  peri¬ 
odic  program  reviews.  For  risk  assess¬ 
ment  to  be  effective,  a  clear  definition 


of  what  constitutes  a  risk  must  exist. 
Table  3.1-3  identifies  criteria  for 
making  this  judgment.  When  a  problem 
area  has  been  identified,  it  must  be 
judged  to  rate  "low"  for  technical, 
schedule,  and  cost  for  it  to  be  rejected 
as  a  risk  item. 

3.1.3  Preliminary  Design  of  TS 

While  preliminary  design  of  a  TS  will 
most  likely  be  accomplished  by  engineer¬ 
ing  specialists  in  the  contractor's 
organization,  the  Air  Force  software 
engineer  will  be  involved  in  at  least  a 
monitoring  and  evaluation  role.  This 
section  describes  how  TS  requirements 
are  derived  during  the  preliminary 
design  activity,  with  particular  empha¬ 
sis  on  software  requirements. 

Preliminary  Design  (PD)  is  a  high-level 
treatment  of  the  simulator  configuration 
allowing  major  interfaces  to  be  identi¬ 
fied  along  with  functional  elements 
responsible  for  the  main  operating  capa¬ 
bilities  required.  Functional  elements 
are  both  hardware  and  software  and  must 
be  considered  together.  General  design 
requirements  on  TS  software  evolve  in 
the  PD  process.  An  example  of  a  highest- 
level  design  for  a  flight  simulator  is 
shown  in  Figure  3.1-9.  This  diagram 
might  result  from  a  ROC  specification 
which  identified: 

a.  Visual  display 

b.  Motion  base 

c.  Operational  cockpit 

d.  Instructor  console 

The  block  diagram  in  Figure  3.1-9  shows 
the  interface  relationship  between  the 
various  functions  and  the  computational 
system.  Once  the  TS  function  and  their 
interfaces  are  defined,  the  computa¬ 
tional  system  can  be  defined.  The  gen¬ 
eral  size  and  capability  of  the  simula¬ 
tion  computer(s)  can  be  established  by 
comparison  with  previously  developed 
systems  and  applying  the  appropriate 
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Figure  3. 1-9.  Block  Diagram— Flight  Simulator 
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factors.  The  general  types  of  computer 
programs  can  also  be  defined  such  as: 

a.  Real-time  operational  programs 

b.  Simulator  support  programs 

c.  Computer  program  system  support 
programs 

d.  Maintenance  and  test  programs 

e.  Calibration  test  programs 

Preliminary  software  design  results  from 
expanding  on  the  basic  software  items 
needed  to  drive  such  a  simulator.  The 
basic  items  are: 

a.  Airplane  dynamics  model 

b.  Airplane  subsystems  models 

c.  Motion  base  drive  program 

d.  Visual  system  drive  program 

e.  Instructor  -  interactive  software 

f.  An  executive  to  manage  program 
interaction  sequence  and  input/output 
through 

(1)  Peripheral  I/O 

(2)  Interface  I/O  routines 

These  are  the  basic  real-time  software 
items  only.  The  ROC  for  the  instructor 
station  might  include,  for  example,  dis¬ 
play  CRT,  mode  control  and  malfunction 
insertion/deletion  capability.  Item  (e) , 
the  Instructor  -  interactive  software 
could  be  depicted  as  in  Figure  3.1-10. 

Suppose  further  that  the  CRT  is  required 
to  display  status  of  the  simulator:  mal¬ 
functions  inserted,  flight  condition, 
flight  subsystem  status  and  other  status 
data.  A  data  base  of  model  parameters, 
control  inputs,  flag,  etc.,  would  be  the 
logical  source  for  data  acquisition.  The 
general  interface  diagram  shown  in  Fig¬ 
ure  3.1-10  implies  I/O  software  and 


intercept  handling.  Adding  these  addi¬ 
tional  capabilities  produces  a  software 
preliminary  design  as  shown  in  Figure 

3.1- 11. 

Figure  3.1-11  is  by  no  means  detailed, 
but  at  this  point  a  software  systems 
engineer  can  begin  to  make  reasonable 
approximations  of  the  size  and  complex¬ 
ity  of  the  software  segments  involved. 
Character-decoding,  I/O  routines  and 
interrupt  handlers  are  well  known; 
"acquire  variables  for  display"  is  an 
unknown  and  probably  a  large  programming 
task.  System  trades  and  their  software 
Impact  can  now  be  made  using  such  a  pre¬ 
liminary  design. 

To  assure  that  all  software  and  hardware 
requirements  are  included  in  the  prelim¬ 
inary  design,  check  lists  were  devised. 
These  are  provided  in  Table  3.1-4  and 
Table  3.1-5  for  software  and  hardware, 
respectively.  Applicable  paragraphs  in 
MIL-D-83468  are  also  noted  for  each  TS 
software  function  in  Table  3.1-4. 

Based  on  the  training/simulation  require¬ 
ments  and  TS  preliminary  design,  the  SPO 
cadre  will  define  several  candidate  TS 
systems  (Figure  3.1-4).  The  type  and  num¬ 
ber  of  candidate  systems  is  influenced 
basically  by  the  background  and  experi¬ 
ence  of  cadre  members  and  augmented  by 
media/equipment  surveys. 

3.1.4  Candidate  System  Selection 

Once  several  candidate  TS  systems  are 
defined  (as  noted  in  the  previous  para¬ 
graphs),  the  SPO  cadre  will  proceed  with 
selection  of  one  candidate  system  on 
which  to  formulate  a  preliminary  TS 
requirements  specification  (Figure 

3.1- 4). 

One  method  for  evaluating  alternate  sys¬ 
tem  configuration  is  discussed  below. 
This  method  identifies  criteria  cate¬ 
gories  for  evaluating  alternate  TS  con¬ 
figurations,  applies  a  weighing  factor 
for  each  category  and  compiles  the 
results  in  a  decision  table  in  which  the 
results  can  be  quantitively  evaluated. 
This  technique  can  be  employed  at  each 
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Figure  3. 1- 10.  Instructor- Interactive  Software 


L. 

CM 

CM 

CM 

CM 

<0 

M 

• 

• 

• 

• 

a. 

00 

CM 

• 

CO 

CM 

« 

CM 

• 

CM 

• 

a>  to 

CM 

* 

CM 

CM 

CM 

r-  «3- 

• 

iH 

• 

• 

• 

n 

10  00 

i  «*» 

• 

CM 

CO 

CO 

CO 

U  1 

* 

• 

* 

9t 

#> 

*r“  Q 

H 

CM 

CM 

CM 

CM 

r-  | 

• 

• 

• 

• 

• 

Ql  —1 

CM 

CM 

CM 

CM 

CM 

TN 

a.p-» 

• 

• 

• 

• 

• 

**> 

S* 

o 

<  X 

CO 

CO 

CO 

CO 

CO 

03  03 

e  c 

o  o 

c  c 


CJ 

03 

03 

p— 

c: 

C 

O 

cn 

• 

+5 

8 

r— 

03 

0 

cn 

L. 

^  c 

U 

CL 

t a 

03 

00 

■r— 

•r“ 

4-> 

cn-r-  ^ 

03 

03 

13 

£ 

•r— 

C 

4J 

r— 

+-> 

09 

c 

c 

O  »“  03 

X 

■a 

S- 

S- 

•r- 

03 

03 

03 

0 

0 

r—  03  +J 

LU 

0 

c 

c 

<u 

4-> 

i- 

C 

-*-> 

+-> 

03 

•r- 

C3 

03  CJ  •»— 

0 

03 

w 

o 

3 

03 

cn 

s- 

O 

> 

4-> 

C7) 

c  to  cn 

9k 

03 

4-> 

*> 

£: 

•r— 

O 

C 

•r— 

03 

s- 

•r— 

S. 

C 

<  T- 

L. 

*a 

c 

95 

CO 

4J 

2: 

i- 

03 

(/) 

> 

S_ 

U9 

03 

■r** 

03 

§■ 

4-> 

o 

jQ 

cn 

c 

■0 

X? 

•r— 

N 

C 

cn 

4-> 

*  to 

u 

03 

x 

c 

3 

_x 

0 

c 

3 

•1— 

O 

c 

C 

cn  •» 

XJ 

03 

<C 

03 

Q 

0 

OO 

c 

u 

a 

03 

O 

cr 

CO 

•r— 

•  r- 

O) 

O  03  tO 

c 

4-> 

X3 

OI 

4- 

•r— 

4-> 

03 

-!-> 

u 

03 

T3 

£ 

1 —  03 

to 

03 

U 

S- 

8 

S- 

U 

o 

4-> 

C 

•n 

0 

c 

03 

.C 

*r~ 

03 

03 

fO  4-> 

U 

03 

03 

oj 

•r* 

03 

O 

4-> 

0 

CO 

+-> 

CO 

■O 

CO 

C 

0 

cn 

C  13  03 

03 

U 

O 

2 

3; 

E 

co 

E 

•r— 

E 

0 

•r— 

u 

03 

C 

u 

to 

c 

■0 

O 

1 

03 

<13  L 

H- 

4-> 

03 

JD 

-»-> 

03 

e 

S_ 

4-> 

-C 

•r— 

01 

■M 

03 

o> 

>> 

•r 

0 

0 

•1“ 

•r— 

c 

^  U 

<4- 

CL 

-C 

>> 

4- 

C 

o 

0 

03 

03 

03 

+-> 

03 

to 

10 

r— 

•r» 

0 

< 

r— - 

03 

f—  1—  to 

3 

3 

u 

03 

J- 

<5% 

O 

50 

•i“ 

4- 

S- 

•r— 

3 

C3 

03 

E 

03 

3 

J- 

03 

O 

2: 

03  *0  "i— 

_Q 

S- 

-X 

03 

LT) 

*o 

4-> 

tO 

cn 

L- 

03 

•1— 

CO 

E 

•f— 

■a 

C 

03 

4-> 

c: 

0 

cn 

S- 

4->  +->  T3 

L. 

-♦-> 

o 

<o 

C 

o> 

O 

1- 

S- 

C 

03 

c 

•t— 

S- 

V 

•r~ 

•r- 

•1— 

4-> 

r— 

•r—  *r— 

03 

03 

3 

E 

J- 

3 

03 

4-> 

COXJ 

-M 

03 

i- 

S- 

3 

cn 

“O 

03 

X) 

TD 

> 

C 

03 

cn  cno 

4-> 

+J 

CL 

0 

C 

03 

cr 

L- 

c 

50 

03 

S- 

03 

O 

0 

c: 

>> 

03 

03 

03 

O 

=3 

•r-  *r-  ^ 

03 

C 

C 

L. 

03 

ft 

LU 

f— 

»— 1 

<c 

sz 

X 

Q.M- 

00 

LU 

m 

LU 

O 

Qi 

2: 

O 

Lu 

O  O  *-• 

*0 

i— t 

» — 1 

tf- 

03 

CO 

03 

to 

03 

> 

•p* 

E 

03 

E 

3  CO 

u 

> 

L- 

4-> 

c 

•r— 

•r- 

-o 

CO 

03  4->  3  +-> 

0 

[ft 

1  ^ 

V- 

>> 

>  CO  Q.M- 

•p 

K 

1  03 

T3 

E 

«/) 

to 

•r-  C  C  CL 

4-> 

c 

C 

03 

•M 

JQ 

l-  -X 

r— 

*r~ 

O 

2? 

03 

4-> 

O 

3 

XJ  0 

O 

X3 

•T~ 

!  ^ 

to 

to 

03 

to 

4->  r-  O 

fc- 

C 

03 

>> 

4- 

ai  •<-  p  u 

4-> 

O 

U 

03 

to 

4- 

03 

u  a  £ 

c 

u 

C 

c: 

E 

E 

03 

C 

03  -X  +>  E 

0 

3 

i  ** 

C  03 

r—  03 

<0 

4-  U  C  O 

u 

Ll. 

0  s- 

03  S- 

O 

L.  O  O  L. 

-C 

'cl 

•«-  cn 

3  C7> 

•r- 

O. 

0J  UU4- 

03 

01 

V- 

+->  0 

to  O 

X3 

u 

4-> 

X3 

•p- 

1  ■>“ 

0  u 

•r-  S~ 

Z3 

•r— 

C  1  1 

O 

f— 

<c 

X  CL 

>  CL 

< 

X 

Ll- 

O  to 

o>  f— 
03  01 


Table  3. 1-5.  Checklist  for  Hardware  Requirements 


I  _ _ _ 

Hardware  Items 


COCKPIT 

•  CONTROLS 

•  DISPLAYS 

MOTION  BASE 

•  HYDR  POWER  SUPPLY 

VISUAL  SYSTEM 
AUDIO  SYSTEM 
INSTRUCTOR  CONSOLE 

I NTERFACE 

•  D/A 

•  A/D 

•  ANALOG 

•  LIGHT  DRIVERS 

•  DISCRETE 

•  DIGITAL  WORDS 

COMPUTER 

•  CPU 

•  MEMORY 

PERIPHERALS 

•  MAG  TAPE 

•  DISC 

•  CR 

•  LP 

•  TPWR 

•  CLOCK 
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level  of  requirement  definition;  i.e. , 
TS  system  computational  system,  computer 
program. 

3. 1.4.1  Criteria  Categories.  The  crite¬ 
ria  for  evaluating  the  candidate  train¬ 
ing  systems  can  be  divided  into  two 
categories: 

a.  Training  Suitability  -  To  what  ex¬ 
tent  does  the  candidate  system  configura¬ 
tion  incorporate  features  which  satisfy 
basic  concepts  of  efficient  learning. 

b.  Support  Paquirements  -  To  what  ex¬ 
tent  does  the "candidate  system  configura¬ 
tion  minimize  requirements  for  unique 
equipment,  personnel,  and  facilities. 

3. 1.4. 1.1  Training  Suitability.  The 
criteria  for  evaluating  each  candidate 
system  as  to  the  extent  that  it  incorpo¬ 
rates  features  which  satisfy  require¬ 
ments  for  efficient  training/simulation 
can  be  divided  into  eight  classes. 

a.  Feedback  -  This  class  of  suit¬ 
ability  criteria  is  concerned  with  the 
extent  to  which  the  candidate  system  pro¬ 
vides  timely  information  to  the  student 
as  to  whether  or  not  his  response  to  a 
specific  stimulus  was  correct.  Correc¬ 
tive  or  re-enforcive  information  may  be 
included  in  this  feedback  such  that  the 
student  learns  from  his  errors  and  his 
successes. 

b.  Participation  -  This  class  of  suit¬ 
ability  criteria  is  concerned  with  the 
extent  to  which  the  candidate  system  pro¬ 
vides  opportunities  for  the  student  to 
engage  in  practice  exercises  throughout 
the  training  cycle. 

c.  Realism  -  This  class  of  suitabil¬ 
ity  criteria  is  concerned  with  the  ex¬ 
tent  to  which  the  candidate  system  is 
judged  to  provide: 

(1)  The  level  of  realism  required 

for  training  of  skills  and  knowledge 

for  each  task  or  subtask,  and 


(2)  The  level  of  realism  required 
for  valid  testing  of  the  student's 
ability  to  perform  the  defined  job 
tasks. 

d.  Self-Pacing  -  This  class  of  suit- 
ability  criteria  is  concerned  with  the 
extent  to  which  the  candidate  system  per¬ 
mits  the  student  to  proceed  through  mul¬ 
tiple  training  exercises  at  his  own 
pace. 

e.  Safety  -  This  class  of  suitability 
criteria  is  concerned  with  the  extent  to 
which  the  candidate  system  configuration 
reduces  the  potential  of  harm  to  stu¬ 
dent,  instructors,  and  equipment  with 
respect  to  actual  job  conditions. 

f.  Response  Recording  -  This  class  of 
suitability  criteria  Ts  concerned  with 
the  extent  to  which  the  candidate  system 
provides  a  record  of  student  responses 
to  training  stimuli.  The  importance  of 
this  factor  is  that  it  provides  the 
instructor  with  a  continuous  basis  for 
diagnostic  of  student  deficiencies  and 
planning  of  remedial  instruction. 

g.  Availability  -  This  class  of  suit¬ 
ability  criteria  is  concerned  with  the 
extent  to  which  the  candidate  system  con¬ 
figuration  reduces  down-line  through 
case  of  maintenance  and  resistance  to 
damage  by  student  use. 

h.  Flexibility  -  This  class  of  suit- 
ability  criteria  is  concerned  with  the 
extent  to  which  the  candidate  system  con¬ 
figuration  lends  itself  to  operating 
demonstrations,  student  practice  ses¬ 
sions,  simultaneous  use  by  students  en¬ 
gaged  in  independent  training  exercises, 
signal  tracking  demonstrations,  and 
other  instructional  uses.  Also  included 
is  the  flexibility  for  updating  of  train¬ 
ing  sequences  in  accordance  with  mission 
system  equipment  and  T.O.  revisions. 

3. 1.4. 1.2  Support  Requirements  Crite¬ 
ria  Classes.  The  criteria  for  evaluating 
each  candidate  training  system  as  to  the 
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extent  it  minimizes  requirements  for  uni¬ 
que  equipment,  facilities  and  personnel 
are  divided  into  five  classes  as  covered 
in  the  following  paragraphs. 

a.  Support  Equipment  -  To  what  extent 
does  the  candidate  system  configuration 
minimize  requirements  for  trainer  unique 
support  equipment  such  as  special  test 
sets  handling  equipment  and  additional 
computer  support  equipment? 

b.  Facil ities  -  To  what  extent  does 
the  candidate  system  configuration  mini¬ 
mize  the  requirements  for  special  facili¬ 
ties  and  services  such  as  special  struc¬ 
tures,  environmental  conditioning.  Radio 
Frequency  Interference  (RFI)  -  proofing, 
and  electrical  power. 

c.  Maintenance  -  To  what  extent  does 
the  candidate  system  configuration  mini¬ 
mize  system  maintenance  requirements 
with  respect  to  number  and  qualifica¬ 
tions  of  personnel,  number  and  type  of 
spares  required,  and  maintenance  flow 
times? 

d.  Computer  Programs  -  To  what  extent 
does  the  candidate  system  configuration 
minimize  the  requirements  for  developing 
and  support  of  unique  computer  programs? 

e.  New  Hardware  -  To  what  extent  does 
the  candidate  system  configuration  mini¬ 
mize  the  requirements  for  developing  uni¬ 
que  equipment  to  be  incorporated  into 
the  training  system? 

Next,  weighting  factors  for  the  eight 
classes  of  training  system  suitability 
criteria  are  selected  on  a  scale  of  1  to 
3  on  the  basis  of  a  review  of  AFP  50-58, 
(Handbook  for  Designers  of  Instructional 
Systems). 

Training  Suitability  Weighting 

Significance  Factor 

(For  Accompl ishing 
Efficient  Training) 


3. 1.4. 2  Training  System  Ratings  and 
Decision  Table.  Each  candidate  system  Is 
rated  on  a  scale  of  I  to  5  as  to  how 
well  it  satisfies  each  class  of  evalua¬ 
tion  criteria.  The  rating  scale  for  this 


evaluation  is  as  follows: 

TRAINING  SYSTEM  EVALUATION  RATING 

a.  Little  or  no  capability  1 

b.  Satisfies  criteria  partially  2 

c.  Satisfies  most  aspects  of  cri¬ 
teria  to  a  satisfactory  level  3 

d.  Satisfies  all  criteria  to  an 

acceptable  level  4 

e.  Satisfies  all  criteria 

exceptionally  well  5 


The  product  of  the  Weighting  Factor  and 
the  Candidate  System  Ratings  are  com¬ 
puted  for  each  candidate  and  then  summed 
for  each  criteria  category. 

The  rating  figures  provide  a  ranking  of 
candidates  system  capabilities  within 
each  category  and  provide  visibility  in 
comparing  the  relative  capabilities  of 
each  candidate  with  respect  to  the  two 
criteria  categories. 

3.1.5  TS  Software  Preliminary  Design 

Software  design  requirements  for  TS  stem 
from: 

a.  Specification  of  TS  functional 
requirements  (RFP  spec), 

b.  Definition  of  software  roles  in  TS 
operations,  and 

c.  Descriptions  of  simulation  events 
and  processes  to  be  performed  or  sup¬ 
ported  by  software. 

Training  simulation  software  will  nor¬ 
mally  be  involved  in  the  processing/ 
presentation  of  sensory  stimuli  and  in 
the  processing/implementation  of  system 
response  to  student  actions.  Software 


Low 

Moderate 

High 


1 

2 
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can  also  serve  in  executive  functions 
controlling  simulation  activity  and  data 
processing/presentation  of  training  per¬ 
formance. 

Unlike  the  requirements  for  the  TS  sys¬ 
tem  and  the  computation  subsystem, 
detailed  computer  program  requirements 
are  derived  by  trade-offs  and  analyses 
conducted  by  a  contractor  during  the  pro¬ 
posal  period  (Figure  3.1.4).  System  and 
computational  system  requirements  are 
derived  by  the  Air  Force  and  documented 
in  the  TS  RFP  system  specification.  The 
recipients  of  the  RFP  system  must  con¬ 
duct  trade-offs  to  determine  which  TS 
function  should  be  allocated  to  hardware 
and  which  to  computer  program  before  com¬ 
puter  program  requirements  can  be 
derived.  These  and  all  trade-offs  must 
be  evaluated  in  light  of  the  basic 
trade-off  criteria  of  cost,  feasibility, 
risk  and  state-of-the-art.  Typically, 
functions  which  require  the  repetitive 
solution  of  a  fixed  relationship  are 
assigned  to  the  special  purpose  proces¬ 
sor.  An  example  of  such  a  function  is 
the  equations  which  simulate  the  flight 
mot  ion/ responses  of  an  aircraft.  Process¬ 
ing  which  is  not  effectively  done  with 
the  special  purpose  processor,  is 
assigned  to  the  general  purpose  computa¬ 
tional  combination  of  hardware  and  soft¬ 
ware. 

When  these  trades  are  completed,  de¬ 
tailed  software  requirements  can  be 
determined  by  the  contractor  and  trans¬ 
lated  into  his  preliminary  design,  which 
is  included  in  his  technical  proposal. 
These  requirements  are  manifested  in  the 
identification  of  computer  program 
modules  \and  a  description  of  the  func¬ 
tion  they  perform. 

Several  examples  of  analyses  conducted 
at  this  level  are  discussed  below. 

3. 1.5.1  Hardware-Software  Trade.  This 
example  assumes  a  flight  simulator 
requiring  a  realistic  force-feel  at  the 
controls  corresponding  to  the  flight 
condition.  The  subject  of  the  trade  is 


whether  the  control  loading  should  be 
accomplished  entirely  by  software  or 
aided  by  some  hardware. 

If  all  the  simulated  control  surface 
positions  and  control -loading  values  are 
to  be  computed  in  the  digital  computer 
as  shown  in  Figure  3.2-12,  it  will  cost 
a  fraction  of  a  millisecond  of  each  simu¬ 
lator  cycle.  The  force-feed  hardware 
will  receive  its  commands  following  a 
large  number  of  digital -to-analog  conver¬ 
sions.  With  some  smoothing  circuitry 
added  to  the  controlling  amplifiers,  the 
controls  will  not  feel  "steppy"  with 
changing  surface  angle.  If,  however,  the 
flight  control  surface  angles  are  com¬ 
puted  with  analog  computer  components, 
the  figure  appears  as  in  Figure  3.1-13. 

Using  this  technique,  a  relatively  few 
words  of  information  from  the  digital 
computer  suffice  to  drive  the  analog  com¬ 
ponents.  The  trade  in  this  case  is  that 
for  the  additional  hardware  cost  of  ana¬ 
log  components,  the  computation  time  for 
control  surface  angles  is  saved. 

Another  example  of  a  hardware-software 
trade  is  the  cost  of  additional  com¬ 
puting  hardware  versus  the  cost  of  com¬ 
pensating  software.  It  is  mentioned  here 
to  highlight  the  fact  that  dollar  costs 
for  off-the-shelf  software  and  hardware 
are  usually  a  minor  part  of  the  total 
system  cost.  If,  for  example,  timing  and 
sizing  studies  show  that  limits  are 
being  reached  or  the  computer  under  con¬ 
sideration,  buying  a  faster,  more  expen¬ 
sive  CPU  and  additional  memory  is  a  rela¬ 
tively  cheap  solution  compared  to  conver¬ 
sion  to  a  lower-level  language,  packing 
data,  employing  sophisticated  overlay 
schemes  and  so  on. 

3. 1.5.2  Malfunction  Insertion/Deletion. 
A  software  program  can  be  provided  (with 
appropriate  interfaces  to  other  simula¬ 
tor  programs  such  as  data  base,  execu¬ 
tive,  etc.)  to  allow  the  instructor  to 
insert  and  delete  malfunctions  from  a 
predetermined  set.  If  an  instructor  dis¬ 
play  is  available,  the  instructor  can 
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Figure  3.1-13.  Flight  Simulator;  Analog-Computed  Surface  Angles 


display  the  current  status  of  malfunc¬ 
tions  in  effect.  These  malfunctions  can 
be  inserted  in  real-time  or  flagged  to 
occur  at  present  times.  Typical  malfunc¬ 
tions  occurring  in  flight  systems  are 
loss  of  hydraulics,  electric  failures, 
etc. 

A  desired  feature  is  to  provide  an  inter¬ 
face  with  the  simulator  executive  rou¬ 
tine  allowing  reset  and  return  to  ini¬ 
tial  conditions  of  normal  functioning. 

3. 1.5. 3  Record/Replay.  A  record/replay 
capability  may  be  specified  in  the  ROC. 
Such  a  program  will  record  on  disc  or 
magnetic  tape,  at  basic  cycle  intervals, 
all  the  contents  of  the  data  base, 
including: 

a.  The  state  vector  completely  des¬ 
cribing  airplane  status 

b.  Control  inputs  from  the  cockpit; 
flight  controls  and  all  pilot  and  flight 
engineer  switches. 

When  playback  is  desired,  through  in¬ 
structor  request,  record/replay  will 
rewind  the  tape  to  the  time  desired  and 
read  into  buffers  the  tape  contents.  By 
means  of  a  logical  "switch"  in  the  soft¬ 
ware,  all  model  control  inputs  can  be 
taken  from  the  recorded  values  in  the 
buffers  instead  of  from  the  cockpit.  The 
airlane  status  will  have  been  initial¬ 
ized  from  the  beginning  point  of  the 
interval  being  played  back.  A  maximum 
time  of  recorded  history  can  be 
specified;  e.g. ,  15  minutes. 

3. 1.5. 4  Display  CRT.  A  CRT  display  for 
the  instructor  station  can  be  specified 
in  the  ROC.  It  may  be  determined  that  an 
auxiliary  computer  is  needed  for  this 
capability,  to  be  driven  by  the  simula¬ 
tion  computer.  The  simulation  computer 
can  transmit  information  for  display  to 
the  auxiliary  computer  which  then  gene¬ 
rates  the  display  symbols  (alphanumeric, 
graphics)  and  refreshes  the  CRT  image. 

The  essential  software  elements  for  the 
simulation  computer  are: 


a.  A  compiler  for  fixed  page  data. 
These  would  be  variable  names,  text, 
borders  and  title  blocks.  The  compiler 
would  run  off-line. 

b.  A  run-time  program  to  fetch  fixed 
page-data  files  from  the  disc  upon  in¬ 
structor  page  request. 

c.  An  update  routine  to  retrieve  the 
current  values  of  variables  and  flags 
from  the  data  base  in  real-time  and  com¬ 
bine  with  the  fixed  page-data. 

d.  A  program  to  transmit  the  data  for 
display  through  a  coupler  to  the  CRT 
mini -computer. 

e.  A  program  to  read  input  from  the 
instructor's  keyboard,  and/or  switches 
requesting  pages  and  malfunction  con¬ 
trol. 

The  essential  software  elements  for  an 
auxiliary  mini-computer  are: 

a.  Application 

(1)  A  run-time  executive  for  pro¬ 
gram  cyclic  control 

(2)  Subroutines  to  generate  the 
alphanumeric  characters  and  graphics 
for  display 

(3)  A  routine  to  decode  input  char¬ 
acter  strings  and  call  subroutines 

(4)  Clocked  image-refreshing  pro¬ 
gram 

b.  Minicomputer  System 

(1)  Assembler 

(2)  Loaders,  bootstraps  and  relo¬ 
catable 

(3)  Source  edit  routine 

(4)  Debug  package 

(5)  Dump  to  printer  program 
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(6)  At  minimum,  a  paper  tape  sys¬ 
tem  with  appropriate  I/O  packages, 
interrupt  handlers. 

The  minicomputer  used  for  CRT  display 
control  can  usually  be  exempted  from 
requirements  for  floating  point  hard¬ 
ware,  a  FORTRAN  compiler  and  a  disc  or 
magnetic  tape  operating  system.  In  gen¬ 
eral,  a  paper  tape  system  with  tele-type 
and  medium-speed  printer  will  be  ade¬ 
quate;  however,  the  balance  of  the  soft¬ 
ware  requirements  of  MIL-D-83468  for  sys¬ 
tems  software  apply. 

3. 1.5. 5  Real-Time  Simulation  Parameter 
Recording.  A  program  can  be  provided  to 
allow  analog  recording,  on  strip  chart 
or  X-Y  plotter,  of  say,  eight  variables 
to  be  selected  from  the  simulator  data 
base.  It  may  not  be  necessary  that  the 
variables  be  selectable  by  typing  in  the 
labels.  Use  of  a  debug  routine  or  a  core 
access  box  to  insert  the  variables  ad¬ 
dress  in  the  appropriate  location  might 
be  acceptable.  Facility  to  scale  the 
variables  for  plotting  will  be  needed. 
The  program  will  likely  make  use  of  the 
standard  I/O  routines  associated  with 
digital-analog  conversion  in  the  simu¬ 
lator.  Storage  on  disc  for  off-line  data 
analysis  is  another  design  alternative. 

This  recording  capability  will  meet  the 
dynamic  test  requirements  in  paragraph 

4.3.10.1  of  MIL-D-83468. 

3.1.6  TS  Software  Trades 

The  contractor  will  perform  a  number  of 
special  software  studies  and  trades  to 
support  TS  software  preliminary  design 
as  input  to  this  TS  proposal  (Figure 
3.1-4).  Some  of  those  studies  which  are 
closely  allied  to  the  TS  configuration 
were  described  in  the  previous  para¬ 
graphs.  Additional  trades  more  specific 
to  software  itself  are  described  in  the 
following  paragraphs.  Examples  follow: 

3. 1.6.1  Simulator  Status  Update.  This 
example  assumes  a  system  requirement 
that  pages  of  simulator  status  shall  be 
displayed  by  flight  systems  upon  command 
on  the  Instructor's  CRT.  These  pages 


shall  contain  lists  of  control  switch 
positions,  cockpit  instrument  values, 
etc.,  labeled  and  updated  in  real  time. 
Assume  further  that  the  requirements 
exist  that  the  Instructor  be  able,  at 
run  time,  to  delete  aqy  entry  on  a  page 
and  replace  it  with  a  different  var¬ 
iable;  there  being  no  requirement  for 
that  replacement  to  be  recorded  back  on 
a  disc  file  as  a  permanent  page  change. 

Preliminary  design  may  indicate  that 
real-time  page  modification  requires 
very  extensive  programming,  consuming  a 
large  amount  of  core  and  running  the 
risk  of  being  impractical  due  to  complex¬ 
ity  and  cyclic  time  constraints.  A  trade 
could  be  effected  in  this  case  wherein 
the  page  changes  are  done  off-line  to 
simulation  operation  (relieving  the 
space/time  problem)  and  permanent  record 
of  the  changes  can  be  saved  on  disc 
files  for  future  retrieval,  making  the 
change  permanent. 

3. 1.6. 2  Software  Implementation  Trade. 
TS  users  and  builders  have  sought  cost- 
effective  means  to  maximize  performance 
realism  and  minimize  simulator  LCC  in 
the  simulation  of  avionics  flight  soft¬ 
ware.  The  potential  is  particulary  great 
where  functions  performed  by  flight  soft¬ 
ware  must  be  duplicated  in  :he  simulator. 

TS  requirements  dictate  that  many  flight 
software  functions  be  reflected  in  the 
simulator.  Simulation  of  controls  and 
displays  requires  processing  equivalent 
to  the  existing  flight  software  if  real¬ 
ism  and  response  time  are  not  to  be  sac¬ 
rificed.  A  majority  of  the  weapon  deliv¬ 
ery,  defense  penetration,  and  navigation 
and  aircraft  steering  functions  done  by 
operational  flight  software  are  also 
applicable  to  the  simulator.  While  some 
reduction  in  simulator  computer  loading 
can  be  achieved  by  simulating  these 
functions,  any  hardware  saving  must  be 
weighed  against  the  high  cost  of  devel¬ 
oping  and  supporting  a  unique  software 
package  for  the  simulation.  Consequently 
a  trade  study  is  conducted  to  determine 
whether  the  simulator  data  processor, 
including  software,  is  an  exact  replica 
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of  the  system  being  simulated  or  whether 
a  different  computer  and  software  suit 
is  to  be  used. 

The  overall  plan  for  this  trade  study  is 
summarized  in  Figure  3.1-14.  The  first 
task  is  to  scope  the  software  elements 
involved  in  the  trade.  Flight  software 
functions  are  evaluated  for  their  rele¬ 
vance  to  training  requirements,  and  for 
the  effectiveness  of  the  flight  software 
modules  in  meeting  the  simulation 
requirements.  The  interfaces  required 
are  defined,  including  hardware  and  soft¬ 
ware.  Interfaces  are  defined  for  incorpo¬ 
rating  the  entire  flight  software  in  air¬ 
craft  hardware  or  an  emulator  or  incorpo¬ 
rating  only  relevant  modules  within  the 
simulator  computer.  Interfaces  include 
not  only  real-world  inputs  to  the  flight 
software,  but  also  simulator  unique 
requirements  for  reset,  freeze,  mode 
switching,  record/playback,  malfunction 
simulation,  initialization,  and  auto¬ 
matic  scoring  and  monitoring.  Avionics 
software  functions  not  required  in  the 
simulator  are  deleted  or  interfaced  to 
not  interfere  with  simulator  operations. 

The  next  study  phase  defines  the  hard¬ 
ware  and  software  configurations  for  the 
indicated  design  options.  This  defini¬ 
tion  includes  all  computer  and  interface 
hardware  required,  operational  flight 
software,  simulation  and  interfacing 
software  required,  and  any  special  sup¬ 
port  hardware  or  software  required  to 
implement  or  support  each  option.  This 
includes  requirements  impact  on  the  sim¬ 
ulation  computer  for  accuracy  and  preci¬ 
sion,  flight  software  iteration  rates, 
timing  synchronization,  and  simulator 
program  data  structures.  Special  soft¬ 
ware  includes  required  compilers,  trans¬ 
lators,  loaders  and  utility  and  debug 
software  to  accommodate  flight  software. 

The  final  study  phase  is  devoted  to 
assembling  the  trade  analysis  data  and 
performing  and  documenting  the  trade. 
Hardware  and  software  design  and  develop¬ 
ment  costs  are  estimated.  Simulation  com¬ 
puter  loading  is  used  to  apportion  simu¬ 
lator  computer  costs  for  each  approach. 
Hardware  and  interface  complexity  is 


evaluated  for  its  effect  on  simulator 
reliability,  maintainability  and  avail¬ 
ability.  Schedule  problems  and  potential 
impact  of  high-risk  items  are  identified 
for  each  option. 

LCC  are  estimated  for  both  hardware  and 
software.  For  software  LCC,  a  simulator 
change  rate  will  be  developed  from  pro¬ 
jections  from  the  system  program  and 
available  simulator  experience.  This  is 
particularly  critical  to  the  trade 
because  the  change  rate  of  flight  soft¬ 
ware  has  high  leverage  in  driving  the 
outcome  of  the  trade.  Hardware  and  soft¬ 
ware  resources  required  to  support  each 
option  are  incorporated  into  the  trade 
data. 

The  cost  trades  are  combined  with  other, 
less  tangible  considerations,  such  as 
risk,  to  arrive  at  a  recommendation. 
Requirements,  features  and  impact  of 
each  option  are  tabulated. 

The  results  of  this  study  provides  basis 
for  selecting  the  most  cost-effective 
method  of  simulation  while  maintaining 
the  necessary  degree  of  training 
realism. 

3. 1.6. 3  Software  Design  Trades.  This 
example  assumes  a  visual  display  system 
is  required.  If  the  simulator  is  planned 
for  night  flight  training,  the  question 
arises  as  to  whether  one  of  the  com¬ 
puter-generated  CRT  displays  would  suf¬ 
fice.  Note  that  in  going  to  a  CRT-type 
display,  a  peripheral  minicomputer  is 
probably  needed  for  refresh  of  the  CRT 
image.  Appropriate  software  to  interface 
between  the  simulator  computer  and  the 
mini  is  needed.  In  trade  for  this  added 
cost  is  relief  from  the  electro-mechan¬ 
ical  complexity  of  image  generation 
using  a  moving  television  camera.  The 
trade  criteria  described  in  paragraph 
3.2.1  can  be  used  to  evaluate  the  two 
options. 

3.2  PLANNING 

Acquisition  of  TS  systems  requires  coor¬ 
dination  and  planning  between  several 
Air  Force  organizations  and  one  or  more 


contractors.  Planning  for  the  software 
development  process  is  contained  in  the 
following  computer  resource  documents: 
(1)  Program  Management  Plan  (PMP),  (2) 
Computer  Resources  Integrated  Support 
Plan  (CRISP),  and  (3)  Computer  Program 
Development  Plan  (CPDP).  Specific  plan¬ 
ning  for  computer  program  requirements 
is  not  a  formally  documented  process  but 
is  integral  to  the  sequence  of  events 
and  timing  of  the  TS  requirements  speci¬ 
fication  process.  The  following  para¬ 
graphs  discuss  the  planning  documents 
supporting  the  requirement  specification 
process. 

3.2.1  TS  Software  Development  Planning 

Planning  for  efficient  use  of  computer 
resources  is  the  responsibility  of  the 
Air  Force.  Normally,  the  Air  Force  will 
prepare  the  PMP  and  the  CRISP,  but  task 
the  CPDP  to  a  contractor  if  a  CPDP  is 
required.  The  following  paragraphs  pro¬ 
vide  a  brief  description  of  the  three 
computer  resources  planning  documents.  A 
more  complete  description  is  found  in 
the  Computer  Program  Documentation 
Requirements  Guidebook. 

3. 2. 1.1  Program  Management.  The  PMP 
provides  comprehensive  planning  for  the 
acquisition  management  of  TS  computer 
resources.  Requirements  for  computer 
resources  evolve  from  overall  system 
requirements  via  application  of  system 
engineering  disciplines.  Computer  re¬ 
sources  are  considered  as  an  integral 
part  of  the  system  and  are  subjected  to 
trade-off  and  optimization  studies  along 
with  other  system  elements.  Refinements 
of  these  studies  through  system  analysis 
result  in  a  set  of  requirements  (speci¬ 
fications)  which  establish  in  detail  the 
required  performance  of  each  system  seg¬ 
ment  and  configuration  item. 

The  PMP  describes  the  system  engineering 
approach  to  be  followed  in  transforming 
operational  needs  Into  computer  resources. 
It  includes:  (1)  a  discussion  of  appro¬ 
priate  trade-offs  between  hardwired  digi¬ 
tal  processing  equipment  and  program¬ 
mable  computers;  (2)  requirements  for 
computer  program  and  data  rights  consis¬ 


tent  with  the  planned  operational  and 
support  concepts;  (3)  a  master  schedule 
of  major  milestones,  key  events,  and 
critical  actions  essential  to  timely 
development  of  computer  resources;  (4) 
requirements  for  acquisition  and  support 
of  documentation;  and  (5)  requirements 
for  simulation  integration  and  necessary 
support  computer  programs.  The  PMP  is 
prepared  by  the  Air  Force  in  accordance 
with  AFR  26-12  and,  together  with  the 
CRISP,  provides  complete  acquisition  man¬ 
agement  and  technical  support  of  com¬ 
puter  resources  over  the  entire  life 
cycle  of  the  TS. 

3. 2. 1.2  Computer  Resources  Integrated 
Support  Plan.  The  CRISP  identifies 
requirements  for  organizational  rela¬ 
tionships  and  responsibilities  for  the 
management  and  technical  support  of  com¬ 
puter  resources  (as  specified  in  AFR 
800-14  Volume  II).  It  functions  during 
the  full-scale  development  phase  to  iden¬ 
tify  computer  resources  necessary  to  sup¬ 
port  computer  programs  after  transfer  of 
program  management  responsibility  and 
system  turnover.  It  continues  to  func¬ 
tion  after  this  transfer  as  the  basic 
agreement  between  the  supporting  and 
using  commands  for  management  and  sup¬ 
port  of  computer  resources. 

The  CRISP  is  written  as  a  part  of  and  in 
parallel  with  the  PMP.  The  CRISP  is  pre¬ 
pared  by  a  Computer  Resources  Working 
Group  (CRWG).  The  CRWG  consists  of  repre¬ 
sentatives  of  the  implementing,  support¬ 
ing  and  using  commands  to  ensure  that 
necessary  elements  of  the  CRISP  are 
included  in  transfer  and  turnover  agree¬ 
ments.  The  CRISP  and  its  periodic 
updates  are  the  responsibility  of  the 
program  manager  and  must  be  approved  by 
him.  The  CRISP  is  developed  during  the 
conceptual  phase  of  TS  system  acquisi¬ 
tion  (prior  to  the  RFP)  and  remains  a 
viable  document  throughout  the  TS  system 
life  cycle.  The  CRISP  is  updated  as 
necessary  to  reflect  changes  in  computer 
resource  requirements. 

3. 2. 1.3  Computer  Program  Development 
Plan.  The  CPDP  is  usually  prepared  by  a 
contractor  for  the  developing  Air  Force 
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agency  and  is  commonly  required  for  a 
proposal.  The  CPDP  may  be  a  contractual 
document  that  applies  to  analysis, 
design,  coding  and  checkout,  test  and 
integration,  and  installation  (if  the 
contractor  is  also  responsible  for  the 
installation  of  the  software). 

The  CPDP  defines  the  contractor's  over¬ 
all  plan  for  developing  the  computer 
programs  and  necessary  supporting  re¬ 
sources.  The  plan  includes  (1)  identifi¬ 
cation  of  the  computer  program  products 
to  be  delivered;  (2)  the  development 
schedule  and  related  documentation;  (3) 
a  description  of  the  contractor  develop¬ 
ment  organization;  (4)  responsibilities 
for  design,  implementation,  testing  and 
integration;  (5)  hardware  and  facilities 
required;  (6)  procedures  for  managing 
and  controlling  all  aspects  of  develop¬ 
ment;  (7)  a  definition  of  the  contrac¬ 
tor's  control  procedures  for  managing 
design  changes  prior  to  the  establish¬ 
ment  of  configuration  management  base¬ 
lines;  (8)  the  reporting  and  management 
of  discrepancies  discovered  in  testing; 
(9)  responsibilities  for  failure  anal¬ 
ysis  and  correction;  and  (10)  retesting 
and  control  of  both  sources  and  object 
code.  If  the  CPDP  becomes  a  contractual 
document,  it  would  then  commit  contrac¬ 
tor  planning  in  development  and  control 
procedures  for  TS  computer  programs.  The 
relationship  of  the  PMP,  CRISP,  and  CPDP 
planning  documents  to  the  process  of 
deriving  TS  requirements  was  'described 
in  the  previous  paragraph  (3.1  -  Techni¬ 
cal  Evaluation). 

3.3  REQUIREMENTS  SPECIFICATION 

As  noted  previously  in  Section  3.0,  the 
end  product  of  requirements  specifi¬ 
cation  is  the  procurement  document 
called  Training  Simulator  Requirements 
Specification,  or  simply  "TS  Specifica¬ 
tion."  Actual  preparation  of  the  TS 
Specification  is  the  subject  of  this 
section  and  the  preparation  is  described 
with  reference  to  technical  evaluation 
(paragraph  3-1)  and  planning  (paragraph 
3.2). 


Also  noted  previously  in  Section  3.0  is 
that  the  principal  task  of  the  AF  TS 
software  engineer  in  requirements  speci¬ 
fication  is  "to  interpret  and  augment 
MIL-D-83468  for  the  specific  TS  system 
being  developed."  This  concept  is  fur¬ 
ther  expanded  in  the  following  para¬ 
graphs. 

3.3.1  Stages  of  TS  Specification  Prep¬ 
aration 

Referring  to  Figure  3.1-4  (paragraph 
3.1),  there  are  three  principal  stages 
to  TS  specification  preparation: 

a.  Preliminary  TS  Specification 

b.  TS  RFP  Specification,  and 

c.  Approved  TS  Requirements  Specifica¬ 
tion. 

The  first  stage  is  preparation  of  a 
draft  TS  specification  based  upon  (1)  TS 
preliminary  design  and  (2)  Definition/ 
selection  of  candidate  system  (see 
Figure  3.1-4).  Also  involved  in  this 
stage  (but  not  shown  in  Figure  3.1-4)  is 
interpretation  of  MIL-D-83468.  The  check¬ 
list  in  Table  3.1-4  can  be  used  conven¬ 
iently  at  this  stage,  but  an  extensive 
item-by-item  evaluation  will  be  employed 
in  the  next  two  stages. 

The  second  stage  is  a  refinement  of  the 
preliminary  TS  specification  to  be  in¬ 
cluded  in  the  TS  procurement  RFP.  Final 
refinement  occurs  in  the  third  stage 
when  the  contractor’s  proposal  has  been 
submitted  and  agreement  is  reached 
between  the  Air  Force  and  TS  contractor 
on  each  item  of  the  requirement  specifi¬ 
cation. 

Supporting  documentation  associated  with 
the  three  stages  is  shown  in  Figure 
3.1-5. 


The  TS  specification  contains  the  re¬ 
quirements  for  all  the  elements  of  the 


3.3.2  TS  Specification  Preparation 
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trainer  system,  including  those  for  the 
hardware  and  computer  programs  which  com¬ 
prise  the  general  purpose  digital  comput¬ 
ational  system.  The  trainer  system  speci¬ 
fication  may  follow  the  format  of  a  Type 
B1  prime  item  development  specification 
as  described  in  MIL-STD-490,  Military 
Standard  Specification  Practices.  This 
B1  specification  is  applicable  to  com¬ 
plex  items  like  aircraft,  missiles,  and 
"training  equipment."  MIL-STD-490  states 
that  this  type  of  specification  must  des¬ 
cribe  effectively  the  detailed  perfor¬ 
mance  that  the  item  is  to  achieve. 

The  first  step  in  preparing  the  prelim¬ 
inary  TS  specification  (1st  stage)  is  to 
determine  what  level  specification  is 
needed,  that  is,  system,  hardware  or 
software.  It  is  possible  that  more  than 
one  level  is  used.  In  the  case  of  a  new 
weapon  system  or  a  new  TS  system  (where 
either  complete  details  of  the  TS  will 
probably  not  be  known  or  a  standard  sys¬ 
tem  has  been  selected),  a  system  speci¬ 
fication  is  appropriated.  By  the  use  of 
the  TS  characteristics  checklists,  the 
specification  can  be  prepared.  The  hard¬ 
ware  (Table  3.1-5)  and  software  (Table 
3.1-4)  checklists  are  provided  in  para¬ 
graph  3.1.3. 

MIL  Spec  (MIL-D-83468)  is  referenced  in 
the  proposed  specification.  Unique  fea¬ 
tures  of  the  subject  TS  can  be  specified 
by  detailed  description  or  stating  devia¬ 
tion/limitation  to  particular  paragraphs 
of  MIL-D-  83468. 

The  functions  to  be  performed  by  TS  are 
stated  first  without  regard  to  their 
implementation,  i.e.,  hardware  or  soft¬ 
ware.  This  includes  the  weapon  system 
functions  to  be  simulated  and  estab¬ 
lishes  the  required  performance  toler¬ 
ances.  It  also  specifies  requirements 
for  a  Training  Director's  console  and 
the  functions  to  be  performed  at  that 
console  including  any  recording  and  play¬ 
back  capabilities.  Computer  programs  are 
required  to  support  the  implementation 
of  these  requirements,  but  detailed 
requirements  for  computer  programs  can¬ 
not  be  specified  at  that  time.  The  ROC 
provides  the  direct  requirement  that 


must  be  satisfied  by  the  system 
specification.  The  design  data  package 
(DDP)  provides  supplementary  data 
regarding  the  characteristics  of  the 
weapon  system  to  be  simulated.  The  ROC 
is  provided  by  an  Air  Force  using  com¬ 
mand  and  is  approved  by  HQ  USAF.  The  DDP 
is  provided  by  the  weapon  system  contrac¬ 
tor.  It  is  either  included  in  the  weapon 
system  Contract  Data  Requirements  List 
(CDRL)  or  is  purchased  directly. 

The  computational  system  is  specified  by 
referencing  MIL-D-83468  Military  Speci¬ 
fication  -  Digital  Computational  System 
for  Real  Time  Training  Simulators.  It 
contains  general  requirements  for  the 
computational  system  equipment  and  the 
Computer  Program  System.  Specific  tai¬ 
loring  of  this  specification  must  be 
performed  to  match  the  particular  TS 
being  developed.  This  specification  is 
not  intended  to  specify  detailed  com¬ 
puter  program  requirements  for  functions 
to  be  simulated.  Rather  it  describes  the 
type  of  computer  programs  that  are 
required  for  the  TS  system. 

Following  the  draft  of  the  specification 
on  the  selected  system,  a  Technical 
Interchange  (TI)  meeting  is  held  with 
all  interested  agencies  participating. 
Both  the  specification  and  study  report 
drafts  are  reviewed.  Following  the  TI 
meeting,  the  comments  approved  by  the 
SPO  should  be  incorporated  in  the  speci¬ 
fication.  When  the  specification  is 
released  in  the  RFP,  both  the  TS  require¬ 
ments  and  weapon  system  DDP  should  be 
part  of  the  package  for  the  contractor 
review  (Figure  3.1-5). 

The  contractor  proposal  is  prepared  in 
response  to  an  RFP  which  contains  the  TS 
system  specification  discussed  in  a  pre¬ 
vious  paragraph.  Upon  RFP  review,  the 
contractor  may  recommend  some  deviations 
to  the  specification.  The  contractor 
technical  proposal  includes  a  prelim¬ 
inary  software  design  for  the  computer 
programs  supporting  the  TS  system 
(Figure  3.1-4).  The  preliminary  design 
is  the  result  of  analysis  and  trade 
study  described  in  paragraph  3.1.5  and 
the  information  obtained  in  the  DDP 
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(Figure  3.1-4).  The  technical  proposal 
identifies  software  modules,  their  inter¬ 
faces  and  describes  the  functions  per¬ 
formed  in  each  module.  It  is  an  explana¬ 
tion  of  detailed  computer  program 
requirements. 

Contractor  proposals  are  evaluated  by 
the  Air  Force  and  the  TS  requirements 
specification  may  be  modified  as  a 
result  of  the  bidder's  proposals  or  as  a 
result  of  contract  negotiations.  In  its 
final  form,  i.e.,  the  result  of  contract 
negotiations,  it  becomes  binding  on  the 
contractor  and  the  Air  Force  and  along 
with  the  contractor  proposal  becomes  the 
equivalent  of  a  development  specifica¬ 
tion.  The  TS  system  will  be  built, 
delivered  and  accepted  in  accordance 
with  this  specification. 

When  the  contractor  proposal  is  approved 
by  the  SPO  and  associated  agencies,  the 
specification  process  is  completed. 

3.4  PROBLEM  AREAS 

The  single  largest  pitfall  in  trainer 
software  development  involves  "add-on" 
capabilities  negotiated  after  require¬ 
ments  have  been  established.  Changes  are 
inevitable  and  become  necessary  when  con¬ 
figuration  changes  occur  to  the  system 
being  simulated.  However,  frequent  or 
untimely  changes  can  cause  significant 
cost,  schedule  and  configuration  control 
difficulties.  These  are  nearly  always 
reflected  back  to  the  government  in  the 
form  of  rising  costs  and  increased  deliv¬ 
ery  flow  times.  These  effects  can  be 
minimized  by  the  following  actions. 

a.  Emphasis  should  be  placed  on 
producing  adequate,  well  thought  out 
requirements  specifications.  This  is 
done  by  identifying  the  requirements, 
all  of  them,  and  thoroughly  analyzing 
alternatives  in  the  manner  indicated  in 
this  guidebook  before  the  specifications 
are  written.  In  this  way,  the  number  of 
changes  can  be  held  to  that  minimum 
consistent  with  real  USAF  requirements. 

b.  Untimely  changes  should  be  avoided 
by  incorporating  changes  at  convenient 


"block"  incorporation  points.  If  pos¬ 
sible,  several  changes  should  be  col¬ 
lected  and  instituted  at  one  time  rather 
than  incorporating  the  several  changes 
independently.  In  this  way  the  frequency 
of  change  is  minimized. 

c.  The  contractor  should  be  made 
aware  of  potential  changes  well  in 
advance  of  their  need  dates  and  his 
advice  solicited  in  matters  concerning 
implementation  of  the  change.  In  this 
way  the  government  benefits  from  the 
contractor's  ability  to  assist  in  plan¬ 
ning  cost-effective  change  incorpora¬ 
tion.  Effective  communication  with  the 
contractor  by  the  TS  acquisition  engi¬ 
neer  should  be  a  continuing  activity 
throughout  the  design,  development  test 
and  production  phases  of  the  contract. 

d.  Additional  problems  related  to  the 
specification  of  TS  software  require¬ 
ments  include: 

(1)  Unnecessary  TS  software  design 
effort  can  result  from  delayed  con¬ 
sideration  of  which  particular  MIL 
Spec  requirements  should  be  exempted 
(for  a  specific  TS  development). 
Exceptions  to  military  specification 
should  be  carefully  analyzed  and  pre¬ 
cisely  stated  prior  to  final  approval 
of  the  procurement  specification. 

(2)  The  impact  of  stated  require¬ 
ments  on  TS  hardware/software  design 
is  often  overlooked.  This  results  in 
costly  system  designs  and/or  subse¬ 
quent  revisions  in  requirements. 
Also,  there  is  a  tendency  to  require 
exacting  performance  of  TS  so  that 
the  best  possible  representation  of 
physical  phenomena  is  attained.  Such 
exacting  requirements  may  not  be 
needed  to  achieve  the  USAF  required 
capability.  Excessively  high  fidelity 
requirements  are  often  very  costly. 
Further  they  may  provide  no  real  bene¬ 
fit  to  the  TS  system.  Each  perfor¬ 
mance  requirement  should  be  scrutin¬ 
ized  carefully  before  a  contracting 
instrument  is  executed,  committing 
the  contractor  to  meet  and  the  govern¬ 
ment  to  pay  for  these  requirements. 
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(3)  TS  system  requirements  pro¬ 
viding  instructor  displays  and  con¬ 
trols  on  simulators,  need  careful 
consideration  and  definition.  For 
example,  more  than  80%  of  the  TS  soft¬ 
ware  development  effort  required  for 
an  advanced  airborne  command  and 
control  system  flight  simulator  was 
expended  in  this  area.  The  cost  fac¬ 
tor  of  TS  instructional  subsystems  is 
so  great  that  particular  emphasis 
should  be  placed  on  TS  system  require¬ 
ments  determination  to  provide  only 
that  minimum  instructional  capability 
consistent  with  USAF  requirements. 
This  effort  alone  can  result  in 
greater  impact  to  TS  software  require¬ 
ments  than  the  combined  effect  of  all 
other  system  requirements.  Advanced 
development  concepts  such  as  real¬ 
time  CRT  instructor  displays, 
i nstructor-machi ne  conversat i onal 
input  output,  etc.,  should  only  be 
specified  when  these  are  clearly 
required  by  the  ROC. 

(4)  Experience  has  shown  that  TS 
procurement  dictates  the  need  for 
clear  definition  of  TS  test  and  veri¬ 
fication  requirements.  A  pitfall  to 
be  avoided  is  the  confusion  caused  by 
unclear  requirements  for  formal  TS 
qualification  testing.  Particularly 
important  is  the  identification  of 
that  testing  activity,  including 
specific  software  tests,  which  are  to 
be  formally  monitored  by  the  USAF. 
The  procurement  specification  should 
not  be  silent  on  this  point. 

(5)  A  frequent  pitfall  is  incom¬ 
plete,  or  improperly  written  Data 
Item  Descriptions  (DID).  This  leads 
to  contractor  misinterpretation  of 
USAF  requirements  and  the  need  for 
unnecessary  revision  of  data  items. 


DIDs  should  be  prepared  in  accordance 
with  normal  practice  for  a  weapon 
system.  Descriptions  should  be 
complete,  yet  concise  and  free  of 
ambiguity. 

3.5  CONCLUSIONS 

Several  major  conclusions  about  the  TS 
software  requirements  derivation  process 
are  listed  below: 

a.  A  systematic  process  for  require¬ 
ments  derivation  does  exist  and  it 
employs  specific  analysis  methods;  trade 
studies;  documentation;  Air  Force  proce¬ 
dures,  and  organization  responsibili¬ 
ties/relationships.  A  composite  overview 
of  the  process  is  provided  in  Figure 
3.1-5. 

b.  TS  software  requirements  cannot  be 
derived  independently  of  TS  hardware  and 
the  derivation  activity  is  a  team 
effort. 

c.  Cost  and  other  development  con¬ 
straints  will  often  dictate  the  use  of 
off-the-shelf  hardware/software  modules 
-  to  be  modified  and  integrated  for  a 
specific  TS  capability. 

d.  A  principal  task  of  the  AF  TS 
software  engineer  is  to  interpret 
MIL-D-83468  for  a  specific  TS  applica¬ 
tion.  Primary  AF  emphasis  is  on  TS  func¬ 
tional  requirements,  whereas  the  TS  con¬ 
tractor  will  conduct  detailed  design 
trade  studies  to  derive  TS  software 
design  requirements. 

e.  Specific  problems  can  be  identi¬ 
fied  in  the  requirements  derivation 
process  but  specific  remedies  can  also 
be  postulated. 
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Section  4.0.  ATE  SOFTWARE  REQUIREMENTS  SPECIFICATION 


Section  4.0  Identifies  and  describes  the 
source  of  ATE  software  requirements; 
describes  the  process  for  specifying  ATE 
software  requirements;  and  provides 
guidelines  for  authorizing  and  moni¬ 
toring  the  specification  of  requirements 
for  ATE  software.  The  term  ATE  refers  to 
the  hardware  and  software  used  for  auto¬ 
matic  testing.  The  hardware  includes 
computing  equipment,  test  adapters  and 
other  test  equipment  used  for  stimulus 
generation  and  measurement.  Software 
includes  the  basic  categories  of  soft¬ 
ware  defined  in  paragraph  4.1.  Much  of 
ATE  software  is  closely  associated  with 
the  test  hardware  and  cannot  be  defined 
separately.  Thus,  the  process  for  speci¬ 
fying  ATE  software  begins  in  the  anal¬ 
ysis  required  for  the  selection  of  ATE 
hardware  even  though  there  is  little  in 
these  analyses  that  is  directly  related 
to  software.  The  process  of  defining  ATE 
begins  with  the  analysis  of  statements 
in  a  weapon  system  ROC  and  continues 
until  the  hardware  is  approved  in  the 
Support  Equipment  Recommendation  Data 
(SERD).  Following  this,  the  process  for 
software  requirements  specification 
begins  and  continues  until  a  development 
specification  (MIL-STD-483),  or  its 
equivalent,  has  been  approved  for  each 
designated  Computer  Program  Configura¬ 
tion  Item  (CPCI).  This  section  is  orga¬ 
nized  accordingly:  (1)  the  impact  of  the 
ROC  and  the  weapon  systems  RFP,  (2)  the 
beginning  of  ATE  requirements  specifica¬ 
tion  in  the  LSA  and  SERO,  (3)  a  descrip¬ 
tion  of  the  process  of  deriving  control 
and  support  software  requirements  and 
the  procurement  of  a  test  set,  and  fin¬ 
ally  (4)  a  description  of  the  specifica¬ 
tion  of  test  software  requirements. 

ATE  software  requirements  stem  basically 
from  two  sources;  from  requirements 
related  to  the  operability  of  the  ATE 
hardware;  and  from  ATE  -  independent 
functional  and  diagnostic  test  require¬ 
ments  of  UUT's  to  be  tested  on  ATE.  How¬ 
ever,  the  selection  of  ATE  itself 
depends  on  projections  of  required  test 
capabilities  of  UUT's  to  be  automati¬ 
cally  tested  including  the  quantity  of 


UUT's  to  be  tested  per  unit  of  time. 
These  requirements  are  directly 
reflected  in  interface  test  adapter 
(ITA)  requirements,  ATE  stimulus  and 
measurement  capability  and  degree  of 
automation  in  testing.  Therefore,  ATE 
software  requirements  specification  is 
part  of  the  total  process  of  identifying 
and  approving  the  support  equipment  for 
a  weapon  system. 

This  paragraph  is  an  idealization  of  the 
sequence  of  events  that  leads  to  ATE 
software  specification.  The  sequences 
described  are  only  generally  true,  and 
are  presented  to  give  a  frame  of  refer¬ 
ence  for  discussion  of  guidelines  and  to 
help  understand  problems.  Events  are  dia¬ 
grammed  in  Figure  4.0-1.  The  weapon 
system  ROC,  the  weapon  system  specifica¬ 
tion,  and  statement  of  work  contain  only 
limited  detail  on  ATE  requirements,  and 
contain  even  less  detail  on  software 
requirements;  thus,  they  are  not  shown 
in  the  figure.  However,  system  deploy¬ 
ment  and  overall  support  concepts  are 
defined  so  that  operational  support 
requirements  may  be  derived.  Basically, 
ATE  procurement  (hardware  and  software) 
depends  on  an  identification  of  the 
operational  support  requirements  for  a 
weapon  system.  The  SERD  is  derived  via 
contracted  LSA,  an  activity  which  is 
usually  part  of  engineering  development. 
Approved  SERD's  are  the  basis  for  prep¬ 
aration  of  prime  item  development  speci¬ 
fications  for  ATE,  which  includes  ATE 
software,  but  not  the  software  needed  to 
test  the  UUT.  Software  for  UUT's  depends 
on  ATE/ITA  design,  and  on  the  perfor¬ 
mance  and  diagnostic  test  requirements, 
which  are  documented  in  the  Test  Require¬ 
ment  Documents  (TRD).  Only  those  require¬ 
ments  in  TRD's  related  to  automatic  test¬ 
ing  are  of  concern  to  the  specification 
of  UUT  software. 

Figure  4.0-1  illustrates  the  essential 
characteristics  of  ATE  software  specifi¬ 
cation.  These  characteristics  are  empha¬ 
sized  in  the  figure  with  heavy  borders. 


a.  ATE  (hardware)  requirements  are 
derived  through  the  LSA  and  documented 
in  a  SERD.  This  process  is  described  in 
paragraphs  4.4  and  4.5,  respectively. 

b.  Control,  support  and  self  test 
software  requirements  depend  on  the 
selection  of  ATE  and  their  ITA's  rather 
than  UUT  design.  Paragraph  4.6  describes 
the  procurement  of  ATE,  including  the 
specification  of  control  and  support 
software  (see  paragraphs  4.1.1  and  4.1-2 
for  definitions). 

c.  ATE  test  software  requirements 
depend  on  the  UUT  designs  and  are  not 
completely  defined  until  all  UUT  designs 
are  completed  and  the  production  models 
are  built.  Performance  and  test  require¬ 
ments  including  test  sequences  are  docu¬ 
mented  in  TRD's.  Paragraph  4.7,  Test 
Software  Requirements,  contains  a  de¬ 
scription  of  the  relationship  of  these 
TRD  data,  the  TRD  and  the  test  software 
development  specification.  The  relation¬ 
ship  of  the  TRD  and  TRD  data  shows  how 
the  lack  of  completely  defined  and 
approved  test  requirements  impacts  the 
development  specification. 

The  SAE  process  for  ATE  software  require¬ 
ments  specification  begins  with  the 
requirements  in  the  weapon  system  ROC 
and  the  weapon  system  RFP  then  continues 
as  illustrated  in  Figure  4.0-1.  It  is 
performed  primarily  by  a  contractor  with 
guidance  provided  by  Air  Force.  The  role 
of  the  Air  Force  ATE  software  engineer/ 
manager*  is  to  monitor  the  requirements 
specification  process,  provide  guidance 
to  the  contracts,  approve  SERD's  for 
support  equipment,  assist  in  the  prep¬ 
aration  of  a  contract  supplement  (if 
necessary)  and  approve  development 
specifications  for  the  computer  programs 
*;o  be  delivered  to  the  Air  Force  along 
with  ATE. 

*"ATE  software  manager/engineer"  refers 
to  a  system  project  officer  who  is 
responsible  to  the  SPO  director  for 
weapon  system  software,  assisted  at 
times  by  engineering  specialists  from 
other  organizations  in  Aeronautical 
Systems  Division  (ASD). 


4.1  ATE  SOFTWARE  DESCRIPTION 

The  three  general  categories  of  ATE  soft¬ 
ware  are  control  software,  support  soft¬ 
ware  and  test  software.  Each  category  is 
defined  in  the  following  paragraphs.  In 
general ,  ATE  control  and  test  software 
operate  together  to  accomplish  UUT 
testing,  while  ATE  support  software 
assists  in  the  development  and  main¬ 
tenance  of  control  and  test  software  by 
providing  such  things  as  language  trans¬ 
lation  capability,  test  station  config¬ 
uration  management  aids  and  program 
development  aids. 

D0DD5000.29  has  been  interpreted  to 
require  that  ATE  software  be  treated  as 
all  DOD  weapon  system  software;  that  is, 
be  subject  to  configuration  management 
per  MIL-STD-483  (and  other  standards) 
and  be  identified  as  one  or  more  CPCI's. 

4.1.1  ATE  Control  Software 

AFLC  Regulation  66-37,  Management  of 
Automated  Test  System,  provides  the 
following  definition  of  control  soft¬ 
ware: 

"Control  software  is  used  during 
execution  of  a  test  program  to  con¬ 
trol  the  non-testing  operations  of 
the  ATE.  This  software  is  used  to 
execute  a  test  procedure  but  does  not 
contain  any  of  the  stimuli  or  measure¬ 
ment  parameters  used  in  testing  the 
unit  under  test  (UUT).  Where  test 
software  and  control  software  are 
combined  in  one  inseparable  program, 
that  program  will  be  treated  as  test 
software,  not  control  software." 

ATE  control  software  is  designed  to  res¬ 
pond  to  test  software  to  enable  test 
functions.  It  also  controls  the  ATE  com¬ 
puter  during  the  conduct  of  a  test.  Its 
source  code  may  be  a  HOL  such  as  FORTRAN 
IV,  but  often  Is  an  assembly  lanauage. 
Interpretive  Abbreviated  Test  Language 
for  All  Systems  (ATLAS)  software  systems 
are  designed  to  accept  ATLAS  test  state¬ 
ments  directly.  The  Interpreter  makes  a 
statement  bv  statement  translation  from 
ATLAS  to  "machine"  language.  In  this 
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case,  the  ATE  control  software  contains 
a  language  interpreter  operating  on¬ 
line.  For  non interpretive  systems,  the 
more  usual  case,  the  ATE  control  soft¬ 
ware  does  not  contain  a  language  inter¬ 
preter  and  ATLAS  statements  must  be 
compiled  to  machine  language.  ATLAS  com¬ 
pilers  may  be  executed  off-line;  i.e. , 
used  at  a  time  other  than  testing,  on 
the  ATE  computer  or  on  a  different 
(host)  computer. 

ATE  control  software  is  usually  mostly 
made  up  of  commercially  developed  soft¬ 
ware  from  a  subcontractor*.  The  remainder 
may  be  newly  developed  or  modified  by 
the  prime  contractor  or  the  subcontrac¬ 
tor. 

Figure  4.1-1  provides  a  typical  example 
of  the  composition  of  ATE  control  soft¬ 
ware.  The  essential  functions  are  an 
operating  system,  a  test  manager,  periph¬ 
eral  drivers,  test  equipment  drivers, 
and  program  development  programs. 

4. 1.1.1  Operating  System  Software.  The 
operating  system  provides  for  control¬ 
ling  and  sequencing  all  programs  to  be 
executed.  It  provides  the  response  to 
all  program  interrupts  and  calls  the 
appropriate  programs  in  response.  A  test 
sequence  will  begin  with  the  operating 
system  initiating  other  control  software 
needed  to  support  the  test  and  will  end 
with  the  operating  system  ensuring  that 
all  functions  are  complete  and  accounted 
for. 

4. 1.1.2  Test  Manager  Software.  Test 
manager  software  controls  the  actual 
sequencing  of  software  test  programs. 
It  operates  when  called  by  the  operating 
system  software.  It  processes  all  opera¬ 
tor  interfaces  and  contains  the  inter¬ 
rupt  processors  associated  with  UUT 
testing. 

4. 1.1. 3  Peripheral  Driver  Software. 
Peripheral  driver  software  controls 
interfaces  to  the  computer  peripherals. 
It  includes  the  programs  to  activate  and 
deactivate  the  data  channels  and  to 


control  the  flow  of  data  to  and  from  the 
peripheral  devices  and  the  computer  main 
memory. 

4. 1.1.4  Test  Equipment  Driver  Soft¬ 
ware.  Test  equipment  driver  software  con¬ 
trols  the  interfaces  to  all  test  equip¬ 
ment  similarly  to  the  peripheral  driver 
software  for  peripheral  devices. 

4. 1.1. 5  Program  Development  Software. 
Program  development  software  provides  an 
on-line  capability  for  software  develop¬ 
ment  and  the  ability  to  make  on-line 
modifications  to  test  or  contro'  soft¬ 
ware.  This  feature  may  or  may  not  be 
included  and  should  be  used  with  dis¬ 
cretion  when  used,  to  prevent  breaches 
in  configuration  management  controls. 

4.1.2  ATE  Support  Software 

ATE  support  software  consists  of  all 
auxiliary  ATE  software  which  is  not 
normally  used  during  the  conduct  of  a 
test.  Though  it  does  not  operate  during 
the  conduct  of  a  test,  it  may  be  resi¬ 
dent  on  the  ATE  computer.  Because  of 
planned  program  utilization  of  the  ATE 
Station  it  is  sometimes  desirable  to 
develop  the  ATE  support  software  using  a 
different  computer.  When  a  host  computer 
is  used  (other  than  the  ATE  computer), 
provision  must  be  made  for  the  support 
software  to  execute  on  the  host  computer 
and  generate  code  for  the  ATE  computer. 
A  compiler  that  is  executed  on  a  host 
computer  and  generates  code  for  another 
"object"  computer  is  called  a  cross 
compiler. 
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Language  translators  are  required  to  con¬ 
vert  the  ATE  source  language  into  ATE 
computer  executable  code  (machine  lan¬ 
guage).  A  means  is  then  required  to  link 
the  code  modules  and  assign  ATE  computer 
memory  locations.  Source  language  in¬ 
cludes  assembly  language  and  HOL,  such 
as  Fortran  IV  and  ATLAS.  Language  trans¬ 
lators  and  linkers  are  typically  a  part 
of  ATE  support  software.  In  addition  to 
translators  and  linkers,  ATE  support 
software  includes  computer  programs 
which  can  be  categorized  as  program 
development  aids  and  test  station  aids; 
e.g.,  a  communications  interface,  and/or 
program  aids,  such  as  an  automatic  test 
pattern  generator  (ATPG). 

ATE  support  software  acquisition  is  simi¬ 
lar  to  that  of  control  software;  i.e., 
commercially-developed  software  is  pur¬ 
chased  from  a  subcontractor  which  may  be 
modified  or  expanded  by  the  ATE  subcon¬ 
tractor  or  weapon  system  prime  contrac¬ 
tor. 

Figure  4.1-2  provides  an  example  of  the 
composition  of  ATE  support  software.  The 
essential  functions  are  language  transla¬ 
tors,  program  development  aids  and  test 
station  aids.  The  functions  under  these 
are  dependent  on  the  specific  ATE  appli¬ 
cation. 

4. 1.2.1  Language  Translators.  Language 
translators  are  required  for  all  com¬ 
puter  program  source  languages  other 
than  machine  instructions.  There  must  be 
a  unique  language  translator  for  each 
computer  and  for  each  language.  If  a 
given  computer  manufacturer  does  not 
provide  that  language  translator  or  it 
has  not  been  developed  previously,  then 
the  language  processor  must  be  devel¬ 
oped.  As  stated  earlier  a  cross  compi- 
ler/assembler  provides  the  capability 
for  translating  computer  program  lan¬ 
guages  for  one  computer  on  a  separate 
host  computer.  This  feature  is  usually 
provided  by  the  weapon  system  contractor 
and  probably  requires  development. 

4. 1.2. 2  Program  Development  Aids.  This 
class  of  programs  includes  all  the  aids 
for  computer  program  development.  Figure 


4.1-2  shows  a  number  of  these.  Some  com¬ 
puter  manufacturers  have  a  number  of 
highly  sophisticated  program  development 
aids  which  can  be  purchased.  Weapon  sys¬ 
tem  contractors  also  may  have  their  own 
set  of  these  aids.  Generally  the  require¬ 
ment  for  the  more  sophisticated  aids  are 
a  function  of  what  is  available  rather 
than  a  hard  requirement  of  need.  Program 
development  aids  may  accelerate  the 
coding  and  checkout  process  of  computer 
programs. 

4. 1.2.3  Test  Station  Aids.  Test  sta¬ 
tion  aids  provide  for  the  mechanics  of 
joining  program  segments  into  an  inte¬ 
gral  unit.  It  may  also  include  computer 
programs  for  automating  configuration 
control  and  computer  program  mainte¬ 
nance. 

4.1.3  ATE  Test  Software. 

Test  software  consists  of  all  software 
used  to  implement  documented  test 
requirements.  It  consists  of  two  types: 
(1)  that  which  is  unique  to  conducting  a 
test  on  a  UUT  with  its  associated  ITA, 
and  (2)  that  which  is  used  to  test  the 
ATE  station;  i.e.,  independent  of  a 
UUT/ITA.  The  latter  test  software  is 
sometimes  called  "self-test”  software, 
but  is  identified  in  this  quidebook  as 
ATE  station  test  software.  Figure  4.1-3 
provides  an  example  of  the  composition 
of  test  software.  The  essential  elements 
are  UUT  test  software,  station  test  soft¬ 
ware,  and  ITA  test  software. 

4. 1.3.1  UUT  Test  Software.  UUT  testing 
is  the  primary  test  station  function.  A 
separate  test  program  must  be  written 
for  each  distinct  configuration  of  UUT. 
The  ATE  with  UUT  test  software  will  be 
used  for  both  performance  (end-to-end/ 
go-no-go)  and  diagnostic  testing.  Perfor¬ 
mance  testing  determines  whether  a  UUT 
is  operating  correctly.  If  the  UUT  does 
not  operate  correctly,  diagnostic  tests 
are  used  to  identify  the  probable  failed 
components.  The  most  common  language 
used  for  test  software  is  ATLAS,  but 
BASIC  and  FORTRAN  have  been  used.  Test 
software  is  usually  developed  by  the 


56 


uLt :  r  .: _  _ 


prime  or  system  contractor;  but  if  sub¬ 
contractors  provide  some  of  the  UUT's, 
they  often  provide  the  appropriate  test 
requirements  information  to  the  prime 
contractor.  In  some  cases,  the  UUT  sub¬ 
contractor  may  develop  tFie  test  software 
for  the  prime'  contractor  when  he  has  the 
qualification  to  do  so. 

4. 1.3.2  ATE  Station  Test  Software.  Sta¬ 
tion  test  software  is  used  to  provide 
confidence  that  the  test  station  will 
perform  as  designed.  In  this  case  the 
test  station  is  considered  the  UUT.  Both 
end-to-end  and  diagnostic  tests  are  per¬ 
formed.  Station  test  software  may  be 
used  for  calibration  purposes  or  for 
maintenance  purposes.  UUT  test  software 
is  independent  of  the  weapon  system  UUT. 
ATE  station  test  software  is  usually 
developed  using  the  ATE  control  software 
source  language. 

4. 1.3. 3  ITA  Test  Software.  ITA  test 
software  is  used  for  the  same  purpose  as 
the  station  test  software.  ITA  test  soft¬ 
ware  could  be  considered  as  part  of  the 
station  test  software  with  the  exception 
of  the  dependence  on  the  UUT.  The  ITA  is 
designed  to  work  with  a  UUT  or  set  of 
UUT's;  therefore,  ITA  test  software  is 
dependent  on  the  UUT.  It  includes  the 
Adapter  Interface  (AI)  files  or  cross 
connection  tables  that  define  the  inter¬ 
face  between  the  UUT  and  the  test  sta¬ 
tion.  ITA  self  test  software  is  designed 
to  test  the  ITA  without  the  UUT  being 
connected.  ITA  test  software  is  usually 
written  in  ATLAS.  If  the  ATE  uses  pro¬ 
grammable  ITA's  (possibly  using  a  micro¬ 
processor)  the  ITA  test  language  will  be 
that  which  is  most  compatible  with  the 
microprocessor  selected  for  the  ITA. 

4.2  REQUIRED  OPERATIONAL  CAPABILITY 

This  paragraph  and  the  following  para¬ 
graph  describe  the  origin  of  general  ATE 
requirements.  Guidelines  for  authorizing 
and  monitoring  the  specification  of  re¬ 
quirements  for  ATE  software  depend  on  an 
understanding  of  the  sources  of  require¬ 
ments,  constraints.  Interfaces,  func¬ 
tions  and  quality  assurance  provisions. 
Most  of  these  development  requirements 


stem  from  the  LSA,  SERD  and  the  TRD,  but 
general  ATE  requirements  are  often  found 
in  ROC's  and  weapon  system  RFP's. 

The  ROC  document  is  a  formal  document 
used  to  identify  an  operational  need  and 
to  request  a  new  or  improved  capability 
for  the  operating  forces.  This  capabil¬ 
ity  is  described  in  terms  of  operational 
objectives,  environment,  support  and 
maintenance  concepts,  and  concept  of 
system  operations.  The  ROC  is  generated 
by  a  using  command  and  approved  by  HQ 
USAF.  Statements  of  requirements  for 
ATE  in  a  ROC  are  usually  very  general. 
Software  requirements  are  probably  not 
identified  at  all.  The  ATE  software 
manager/engineer  is  usually  not  involved 
in  either  the  generation  or  review  of  a 
weapon  system  ROC. 

Sometimes  a  ROC  may  be  issued  specifi¬ 
cally  to  procure  an  ATE  capability.  In 
this  instance  the  ATE  software  manager/ 
engineer  may  be  invited  to  participate 
in  the  development  of  the  ROC.  He  will 
then  provide  the  technical  assistance 
requested.  An  example  of  a  ROC  for  ATE 
is  a  ROC  issued  for  a  Central  Air  Data 
Computer  test  set.  Even  in  this  case  ATE 
software  requirements  are  probably  mini¬ 
mal. 

4.3  WEAPON  SYSTEM  RFP 

An  RFP  for  a  major  weapon  system  is 
issued  to  a  bidders  list  of  prospective 
contractors.  The  elements  of  the  RFP 
that  are  of  concern  for  ATE  software 
requirements  specification  are  the  State¬ 
ment  of  Work  (SOW),  the  Contracts  Data 
Requirements  List  (CDRL)  and  the  System 
Specification.  The  CDRL  is  of  interest 
because  it  may  specify  requirements  for 
computer  program  development  specifica¬ 
tions.  A  more  thorough  description  of 
the  CDRL  Is  found  in  the  Computer  Pro¬ 
gram  Documentation  Requirements  guide¬ 
book.  Requirements  for  ATE  could  appear 
In  the  SOW  and  the  system  specification. 
The  extent  to  which  requirements  are 
specified  in  a  weapon  system  RFP  depends 
on  the  contracting  method  used.  ATE  may 
be  acquired  by  direct  Inclusion  In  the 
SOW  and  the  system  specifications,  it 
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may  be  acquired  by  a  contract  supplement 
to  the  prime  contract  or  it  may  be  ac¬ 
quired  by  a  separate  contract.  The  first 
and  third  methods  are  not  the  usual,  al¬ 
though  some  current  weapon  system  con¬ 
tracts  are  using  the  direct  inclusion 
and  there  are  always  isolated  instances 
when  an  ATE  capability  is  contracted 
separately  for  an  existing  weapon  sys¬ 
tem.  Requirements  specification  for  all 
the  methods  is  similar  and  is  included 
in  paragraph  4.6.  ATE  acquisition  by 
contract  supplement  is  the  usual  method 
and  is  addressed  in  this  paragraph. 

Since  the  weapon  system  RFP  does  not 
specifically  address  ATE  requirements, 
the  responsibilities  of  the  ATE  software 
manager/engineer  for  ATE  software  re¬ 
quirements  specification  are  limited. 
The  SOW  must  include  the  tasks  from 
which  ATE  software  requirements  are  con¬ 
ceived.  They  are  the  LSA  tend  the  genera¬ 
tion  of  the  SERD.  These  items  are  dis¬ 
cussed  in  the  two  following  paragraphs. 

The  question  of  whether  to  include  ATE 
in  weapon  system  RFP  is  the  subject  of 
trade-off.  Including  ATE  requirements 
provides  an  emphasis  on  the  ATE  task  and 
provides  for  long-lead  planning.  ATE  has 
typically  been  de-emphasized  during  the 
early  stages  of  a  weapon  system  develop¬ 
ment  and  then  received  much  attention 
when  its  use  is  imminent.  On  the  other 
hand,  much  of  the  ATE  and  ATE  software 
is  dependent  on  the  UUT's  which  are 
years  from  development.  This  long-time 
lag  may  invalidate  the  ATE  requirements 
in  the  weapon  system  RFP  and  cause  a  con¬ 
siderable  amount  of  rework  and  change  ac¬ 
tivity.  As  stated  earlier,  both  methods 
are  being  used.  The  trade-offs  must  be 
evaluated  for  the  specific  application. 
If  ATE  requirements  are  not  included, 
the  SOW  should  make  provision  for  a  plan¬ 
ned  Contract  Change  Proposal  (CCP)  for 
augmenting  the  prime  contract  at  the 
appropriate  time. 

4.4  LOGISTICS  SUPPORT  ANALYSES 

The  LSA  is  a  group  of  related  tasks  or 
analyses  performed  by  the  prime  con¬ 
tractor.  The  objectives  of  the  LSA  are: 
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a.  Develop  requirements  for  support 
resources. 

b.  Assure  that  the  support  system  con¬ 
straints  are  reflected  in  the  weapon  sys¬ 
tem  design. 

c.  Integrate  the  various  logistics 
activities  by  maintaining  a  centralized 
source  of  logistics  data  for  use  by  all 
the  specialty  areas  in  logistics. 

d.  Provide  logistics  management  data 
to  the  prime  contractor  and  Air  Force 
logistics  managers. 

MIL-STD-1388-1  describes  what  the  LSA 
must  include.  It  does,  however,  permit 
the  prime  contractor  and  the  SPO  to 
implement  the  LSA  in  a  manner  that  they 
feel  is  appropriate  to  the  procurement. 

Most  prime  contractors  have  their  own 
worksheet  formats  and  computer  programs 
for  summarizing  the  data  produced.  For 
those  who  do  not,  the  Ariqy  has  developed 
the  computing  software  and  worksheets 
and  will  provide  these  at  no  cost.  The 
Army  approach  adheres  closely  to 
MIL-STD-1388-1. 

A  simplified  representation  of  the  LSA 
process  is  shown  in  Figure  4.4-1.  The 
task  analysis  (Block  1)  is  the  central 
portion  of  the  LSA.  ft  provides  a  break¬ 
down  of  tasks  required  to  accomplish  all 
maintenance  and  general  support  for  the 
equipment  item.  For  each  such  task,  the 
following  data  are  recorded: 

a.  Brief  description  of  task 

b.  Frequency  of  occurrence  of  the 
task 

c.  Task  duration  or  time-to-accom- 
plish 

d.  Recommended  location  for  the  task 

(1)  Flightline 

(2)  Field  shop  (intermediate 

level ) 
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Figure  4.4- 1.  Logistics  Support  Analysis  (LSA)  Process 
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(3)  Depot  or  contractor-furnished 
e.  Resources  required 

(1)  Personnel  (quantity  and 
skills) 

(2)  Fools,  test  equipment,  and 

other  handling  or  support  equipment 

(3)  Spares,  repair  parts,  and 

maintenance  materials 

(4)  Facilities 

The  Task  Analysis  is  a  "think-through" 
of  the  maintenance  or  support  task  by  an 
engineer  who  is  thoroughly  familiar  with 
maintenance  of  that  type  of  equipment. 

The  first  step  is  to  identify  the  tasks 
to  be  analyzed.  The  kinds  of  tasks  to  be 
identified  are  classified  as  follows: 

a.  Corrective  Maintenance 

(1)  Fault  localization  and 
isolation 

(2)  Remove  and  replace  defective 
unit 

(3)  Repair  defective  unit 

(4)  Adjust  or  align 

(5)  Checkout  after  repair 

b.  Preventive  Maintenance 

(1)  Inspect 

(2)  Replenish  fluids 

(3)  Periodic  replacements 

c.  Support  General 

(1)  Load  and  unload  weapons, 
cargo,  payload 

(2)  Ground  transportation  and 
handling  of  payload 


(3)  Missionization  of  the  air 

vehicle 

(4)  Towing,  parking  air  vehicle 

(5)  Fueling,  defueling  air  vehicle 

Corrective  maintenance  tasks  are  identi¬ 
fied  with  the  aid  of  the  contractor's 
failure  modes  and  effects  analysis 
(FMEA)  -  Block  4,  Figure  4.4-1.  Corres¬ 
ponding  to  each  failure  mode  there  are 
one  or  more  corrective  maintenance 
tasks.  Preventive  maintenance  tasks  are 
identified  by  other  entries  on  a  FMEA 
worksheet,  such  as  "Life"  and  "mean- 
time-between-overhauls"  (MTBO).  The  prime 
contractor's  reliability  program  devel¬ 
ops  the  data  for  FMEA.  The  support  gen¬ 
eral  tasks  to  be  analyzed  are  identified 
with  the  aid  of  the  use  studies  (Block 
2,  Figure  4.4-1).  This  LSA  task  is  con¬ 
cerned  with  the  way  the  Air  Force  in¬ 
tends  to  use  the  weapon  system,  the  con¬ 
cept  for  use  or  employment  concept.  The 
outputs  of  the  use  studies  are  (1) 
expanded  functional  flow  diagrams  of 
ground  operations  and  maintenance  activ¬ 
ities,  and  (2)  support  planning  factors. 
Support  planning  factors  include  the 
following  data  about  support  general  and 
mission  elements  planned  for  the  air 
vehicle: 

a.  Frequencies  of  occurrence 

b.  Durations 

c.  Locations 

d.  Contingency  operations 

The  third  LSA  task  is  a  historical  data 
review  (Block  3,  Figure  4.4-1).  The 
prime  contractor  obtains  experience  data 
from  the  Air  Force  on  similar  air  vehi¬ 
cles  that  will  help  him  identify  ele¬ 
ments  of  the  equipment  that: 

a.  Are  high  failure-rate  items 

b.  Are  downtime  contributors 
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c.  Present  safety  problems 

d.  Are  support  cost  drivers 

e.  Present  gross  requirements  for  sup¬ 
port  resources 

The  prime  contractor's  maintainability 
program  (Block  5,  Figure  4.4-1)  will 
provide  estimates  of  the  amount  of  time 
required  to  perform  most  of  the  tasks  to 
be  analyzed.  Other  task  duration  data 
will  result  from  the  use  studies  (Block 
2,  Figure  4.4-1)  and  the  historical  data 
review  (Block  3,  Figure  4.4-1). 

Maintenance  task  frequencies  will  result 
from  the  prime  contractor's  reliability 
program  (Block  4,  Figure  4.4-1)  and  the 
historical  data  review  (Block  3,  Figure 
4.4-1).  Support-general  task  frequencies 
are  contained  in  the  support  planning 
factors  produced  by  the  use  studies 
(Block  2,  Figure  4.4-1). 

An  optimum  repair  level  analysis  (ORLA) 
is  an  examination  of  an  equipment  item 
to  establish  the  best  location  for  re¬ 
pairing  it  when  it  fails  (Block  6, 
Figure  4.4-1).  The  alternatives  are: 

a.  Discard  the  item  on  failure 

b.  Repair  the  item  at  the  field  shop 
or  intermediate  level 

c.  Repair  the  item  at  the  depot  level 

All  of  the  failure  modes  are  examined, 
considering  economic,  operational,  and 
other  constraints.  ORLA  reports  to  the 
SPO  include  recommended  repair  level  and 
the  criteria  and  rationale  used  in  arriv¬ 
ing  at  the  recommendation.  The  procedure 
for  performing  the  ORLA  is  described  in 
AFLCM/AFSCM  800-4,  "Optimum  Repair  Level 
Analysis  (ORLA)". 

Locations  for  performing  the  repair 
tasks  are  provided  by  the  ORLA  (Block  6, 
Figure  4.4-1).  Locations  for  performing 
the  support-general  tasks  are  produced 
by  the  use  studies. 


As  the  task  analysis  portion  of  the  LSA 
proceeds,  problems  will  usually  come  to 
the  surface;  e.g. ,  need  for  test  points 
that  had  not  been  planned.  These  are  fed 
back  to  the  designers  so  that  corrective 
action  can  be  taken.  In  some  cases,  the 
best  alternative  may  not  be  obvious, 
requiring  that  a  trade  study  be  con¬ 
ducted.  To  support  the  trade  study,  the 
LSA  team  may  require  that  the  alterna¬ 
tives  be  costed  by  the  contractor's  LCC 
activity.  One  of  the  most  widely  used 
LCC  models  is  the  LSC  model  developed  by 
the  Air  Force.  In  performing  the  trades, 
other  potential  gains  and  losses  must  be 
considered;  e.g., 

a.  Transportability 

b.  Reliability 

c.  Maintainability 

d.  Safety 

e.  Performance 

f.  Schedule 

The  design  feedback,  LCC  analysis,  and 
trades  loop  are  shown  in  Figure  4.4-1  as 
Blocks  7,  8  and  9. 

The  maintenance-planning  task  (Block  10, 
Figure  4.4-1)  starts  with  the  mainte¬ 
nance  concept  and  expands  it  into  a  main¬ 
tenance  plan.  The  maintenance  plan  forms 
the  basis  for  tracking  the  other  ele¬ 
ments  of  logistics.  Initially,  the  main¬ 
tenance  plan  is  made  up  of  concepts, 
goals,  and  constraints.  As  the  LSA  pro¬ 
gresses,  the  maintenance  plan  draws 
together  the  story  of  how,  when,  where 
and  by  whom  the  maintenance  of  the  air 
vehicle  will  be  done.  Scheduled  mainte¬ 
nance  requirements  should  be  planned 
using  the  methodology  outlined  in  the 
appendix  of  MIL-M-5096D. 

Among  the  most  Important  outputs  of  the 
LSA  are  the  descriptive  requirements  for 
support  resources.  They  are  developed 
during  the  task  analysis  portion  of  the 
LSA.  Examples  of  these  descriptive 
requirements  follow: 


a.  Support  Equipment:  Test  Amplifier 
input/output  voltage,  center  frequency 

b.  Support  Facility:  220  3-phase 
power  supply 

c.  Technical  Data:  Amplifier  voltage 
level,  center  frequency  profile 

d.  Spares:  Amplifier 

e.  Personnel,  Skills:  Radar  Repair 

It  is  level  of  detail  in  descriptive 
requirements  that  are  of  prime  impor¬ 
tance  to  the  ATE  Engineer. 

4.5  SUPPORT  EQUIPMENT  RECOMMENDATION 
DATA 

The  role  of  the  SERD  in  bridging  the  gap 
between  the  LSA  and  the  ATE  software 
specification  is  shown  in  Figure  4.5-1. 
Referring  to  that  figure.  Block  1  shows 
the  task  analysis  portion  of  the  LSA,  as 
described  in  paragraph  4.4.  Block  2 
shows  one  type  of  output  from  the  task 
analysis;  i.e. ,  descriptive  requirements 
for  support  equipment,  and,  in  this 
case,  descriptive  requirements  for  test 
equipment. 

Each  task  analyzed  by  the  LSA  is  asso¬ 
ciated  with  an  item  of  equipment.  That 
item  of  equipment;  e.g. ,  an  avionics 
unit,  to  be  tested  will  be  called  a  UUT. 
Block  2  in  Figure  4.5-1  shows  UUT- 
oriented  descriptive  requirements  for  a 
task  being  described  by  the  LSA  to  be 
measured  and  first  estimate  of  expected 
values. 

The  LSA  helps  the  contractor's  support 
equipment  activity  to  pull  together  or 
aggregate  (Block  3)  these  requirements 
for  one  or  more  UUT's  to  develop  a  recom¬ 
mended  aggregate  solution.  The  summary 
of  descriptive  requirements  Is  entered 
on  the  first  part  of  the  SERD.  The  recom¬ 
mended  solution  Is  on  the  second  part  of 
the  SERD.  Finally,  the  last  part  must 
contain  procurement  data  such  as  prices, 
quantities,  and  location.  DID  DI-S-6176 


describes  the  SERD  once  known  as  the 
Aerospace  Ground  Equipment  Requirements 
Documentation  (AGERD). 

A  SERD  is  prepared  for  each  support 
equipment  or  ATE.  When  adapters  are 
required,  the  SERD's  are  prepared  for 
those  also.  In  some  cases,  a  SERD  might 
accommodate  a  set  of  adapters  (see  Block 
4  and  5  of  Figure  4.5-1). 

Flow  time  for  Air  Force  approval  of  the 
SERD  should  be  less  than  two  months.  If 
the  flow  time  exceeds  that,  then  the  con¬ 
tractor  may  assume  the  SERD  is  tacitly 
approved  (see  Block  6,  Figure  4.5-1). 
Approval  of  the  SERD  authorizes  the 
prime  contractor  to  proceed  to  develop 
ATE  specifications  (in  the  case  of  new- 
development  ATE)  or  to  initiate  purchase 
(in  the  case  of  off-the-shelf  ATE).  Con¬ 
trol  and  support  software  development 
specifications  are  generated  and  the 
test  status  is  procured  (Block  11,  Fig¬ 
ure  4.5-1). 

The  UUT-oriented  descriptive  test  re¬ 
quirements  (Block  2,  Figure  4.5-1)  are  a 
collection  of  parameters  and  values  to 
be  tested.  These  tests  are  not  se¬ 
quenced.  Block  7  shows  the  next  step, 
sequencing  the  tests  for  the  UUT.  This 
defines  the  performance  test  or  "go- 
path." 

UUT  requirements  aggregated  to  one  adapt¬ 
er  (or  adapter  set)  are  grouped  in  order 
to  develop  diagnostic  or  fault-isolation 
test  sequences  (Block  8,  Figure  4.5-1). 

The  UTT-oriented  descriptive  require¬ 
ments  for  test  (Block  2),  the  UUT  perfor¬ 
mance  test  sequences  (Block  7)  and  the 
diagnostic  test  sequence  (Block  8)  make 
up  a  significant  part  of  the  TRD  for  the 
UUT.  TRD's  are  prepared  in  accordance 
with  MIL-STD-1519  for  all  avionics 
items.  The  total  set  of  ATE  test  soft¬ 
ware  requirements  are  comprised  of  the 
TRD  data  (Block  9)  and  the  ATE  test  soft¬ 
ware  development  specification  (Block 
10).  This  Is  a  highly  simplified  descrip¬ 
tion  of  the  software  requirement  specifi¬ 
cation  process.  Paragraphs  4.6  and  4.7 
provide  a  more  complete  description. 
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Figure  4.5-1.  SERD  Interfaces  With  LSA  and  ATE  Software  Requirements 
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There  Is  an  orderly  methodology  from  the 
task  analysis  to  descriptive  require¬ 
ments  to  the  SERD  to  ATE.  If  ATE  alter¬ 
natives  or  options  are  precluded  by 
forcing  the  contractor  to  select  ATE  too 
early,  then  the  ATE  cost-effectiveness 
picture  Is  compromised. 

The  LSA  and  the  generation  of  the  SERD 
are  contractor  activities  specified  in 
the  weapon  system  SOW  and  the  CDRL.  The 
Air  Force  ATE  software  manager/engineer 
must  keep  abreast  of  the  LSA  and  be  cog¬ 
nizant  of  the  studies  being  performed. 
The  primary  output  is  the  SERD  which 
defines  (for  our  purposes)  the  required 
ATE.  Each  SERD  must  be  approved  by  the 
Air  Force  within  a  60  day  period.  The 
SERD  must  be  reviewed  in  light  of  the 
studies  and  reports  resulting  from  the 
LSA  and  approved  or  disapproved  accord¬ 
ingly.  The  approval  of  ATE  is  the  real 
beginning  for  the  requirements  specifica¬ 
tion  process  for  ATE  software. 

Usually  there  are  only  a  few  ATE  soft¬ 
ware  manager/engineers  available  for 
monitoring  the  LSA  process.  This  tends 
to  place  the  Air  Force  at  a  disadvantage 
as  the  contractor  will  employ  a  number 
of  experts  in  the  analysis.  Care  must  be 
taken  not  to  overlook  this  phase  and  to 
use  experienced  qualified  personnel  for 
monitoring  the  LSA  process. 

4.6  ATE  PROCUREMENT 

Procurement  of  ATE  can  begin  after  the 
SERDs  are  approved  by  the  Air  Force.  The 
SERDs  define  the  ATE  that  is  approved 
for  the  weapon  system.  Computer  program 
requirements  have  not  yet  been  speci¬ 
fied.  This  section  will  focus  on  the 
requirement  specification  of  ATE  control 
and  support  software,  providing  guidance 
for  beginning  ATE  software  requirements 
activity;  the  contracting  method  (a  con¬ 
tract  supplement  in  this  case);  specifi¬ 
cation  of  control  and  support  software 
requirements,  the  subcontract  for  the 
test  station  and  the  role  of  the  Air 
Force  ATE  software  manager/engineer. 


4.6.1  Beginning  ATE  Software  Require¬ 
ments  Specification 

ATE  is  intended  to  be  used  as  opera¬ 
tional  support  equipment,  dealing  with 
production  equipment  and  as  production 
acceptance  test  equipment  and  procedure. 
Therefore,  as  a  rule  of  thumb,  efforts 
for  the  requirements  specification  of 
ATE  software  should  be  initiated  two 
years  before  the  scheduled  delivery  of 
first  UUT  production  units.  The  UUT 
source  data  required  to  specify  ATE 
Station  stimuli  and  sensors  (and  their 
performance  requirements)  and  to  ini¬ 
tiate  UUT  test  software  and  ITA  devel¬ 
opment,  would  be  of  questionable  quality 
if  demanded  too  soon.  Research  Design 
Test  and  Evaluation  (RDTSE)  design  evolu¬ 
tion  causes  changes  in  the  UUT  designs 
and  their  associated  test  requirements. 
Also,  ATE  technology  is  developing  at  a 
rapid  rate  and  it  is  desirable  to  take 
advantage  of  the  latest  technology  fea¬ 
sible  for  the  weapon  system  to  be  devel¬ 
oped.  The  two-year  flow  time  provides 
adequate  time  to  phase  ATE  hardware  and 
software  requirements  development. 

Identification  of  the  earliest  availabil¬ 
ity  of  the  production  configuration  UUT 
dedicated  for  ATE  system  development, 
determined  that  point  during  a  program 
when  a  UUT  has  adequate  design  maturity. 
ATE  station  procurement  can  then  be  plan¬ 
ned  and  scheduled  as  shown  on  Figure 
4.6-1.  "Zero"  time  on  the  chart  is  the 
start  of  UUT  test  software  specifica¬ 
tion.  This  figure  shows  flow  time  keyed 
to  a  point  at  +1  years,  which  is  the 
availability  of  the  first  production  UUT 
dedicated  for  ATE  system  development.  At 
this  point  software  integration  and  soft- 
ware-hardware  demonstration  tests  can  be 
accomplished.  Backing  up  in  time  from 
this  point,  ATE  station  procurement  is 
shown  together  with  normal  acquisition 
steps. 

A  scheduling  conflict  is  noted  when  com¬ 
paring  the  target  date  for  ATE,  Station 
Stimulator/Sensor  requirements  data  and 
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the  availability  of  TRD  input  data  (par¬ 
ticularly  diagnostic  data).  From  a  stand¬ 
point  of  true  need,  these  schedules 
should  be  reversed.  TRD  input  data  are 
needed  to  specify  characteristics  of  the 
ATE  station  stimuli;  sensors,  ATE  con¬ 
trol  software  timing;  and  ATE  station 
resources  allocation.  Despite  this  con¬ 
flict,  Figure  4.6-1  shows  ATE  software 
requirements  development  and  procurement 
in  two  phases.  The  first  phase  addresses 
ATE  control  and  support  software,  i.e. , 
part  of  the  ATE  station  package.  The 
second  stage  will  address  UUT  test  and 
ITA  test  software. 

4.6.2  Contract  Supplement 

Following  approval  of  the  SERDs,  some 
method  of  contracting  for  ATE  procure¬ 
ment  is  usually  needed.  Some  form  of 
contract  supplement,  such  as  a  CCP,  is 
commonly  used.  Other  methods  were  identi¬ 
fied  in  paragraph  4.3.  A  detailed  discus¬ 
sion  of  contracting  for  ATE  computer  pro¬ 
gram  is  provided  in  the  "Contracting  for 
Software  Acquisition"  guidebook. 

At  this  point  the  project  office  usually 
will  request  the  weapon  system  contrac¬ 
tor  to  prepare  an  amendment  to  the  weap¬ 
on  system  SOW  for  the  inclusion  of  ATE. 
The  number  of  Air  Force  ATE  software  man¬ 
ager/engineers  assigned  to  a  project  is 
usually  severely  limited.  Therefore, 
their  participation  in. preparing  the  SOW 
for  the  CCP  is  monitoring  the  activities 
of  the  contractor  and  providing  techni¬ 
cal  consultation  in  defining  the  tasks 
related  to  ATE  software  requirements 
specification  and  software  development. 
The  SOW  should  define  the  tasks  of  soft¬ 
ware  requirements  specification  and  spec¬ 
ify  the  requirement  for  a  computer  pro¬ 
gram  development  specification  for  each 
CPCI.  In  addition,  the  SOW  should  spec¬ 
ify  the  applicability  of  an  ATLAS  lan¬ 
guage  specification.  The  AF  ATE  engineer 
should  ensure  that  the  SOW  states  that 
UUT  test  software  and  ITA  Self-Test 
shall  be  done  in  ATLAS  in  accordance 
with  an  identified  ATLAS  language  speci¬ 
fication.  The  SOW  should  state  that  a 
HOL,  preferably  FORTRAN  IV  be  used  when¬ 
ever  possible  in  any  newly  developed 


control  and  support  software  and  that 
assembly  language  be  used  only  when  it 
is  impractical  or  impossible  for  a  pro¬ 
gram  coded  in  the  HOL  to  satisfy  the  pro¬ 
gram  requirement.  Additional  considera¬ 
tions  to  be  included  in  the  SOW  are  as 
follows: 

4.6. 2.1  Security  Provisions.  When  the 
ATE  software  is  required  to  process 
classified  information,  the  contractor 
should  include  in  his  proposal  the  ad¬ 
ministrative,  physical,  and  personnel 
security  measures  required  to  protect 
the  classified  material,  together  with 
his  plans  for  implementing  these 
measures. 

4. 6. 2. 2  Support  Software  Training  Re¬ 
quirements.  If  a  significant  amount  of 
ATE  support  software  is  anticipated,  it 
may  be  appropriate  to  direct  the  con¬ 
tractor  to  provide  an  estimate  of  the 
requirements  and  recommended  approach 
for  training  the  personnel  needed  to 
develop  ATE  support  software. 

4. 6. 2. 3  Data  Rights.  The  Air  Force, 
particularly  AFLC,  may  anticipate  that 
it  will  have  further  need  of  computer 
programs  or  data  generated  under  the 
contract.  In  this  situation,  a  statement 
stating  that  the  contractor  is  required 
to  give  up  proprietary  rights  to  the 
subject  computer  programs  is  required. 

4.6. 2.4  Nuclear  Safety.  AFR  122-10 
states  that  the  software  used  for  test¬ 
ing  or  checkout  of  aircraft  or  missile 
systems  employing  nuclear  weapons  must 
meet  certain  safety  criteria.  Specifi¬ 
cally,  122-10  states  that  any  software 
which  can  exercise  automated  or  auto¬ 
matic  control  over  any  critical  nuclear 
weapon  system  function  must  be  subjected 
to  a  software  Nuclear  Safety  Cross-Check 
Analysis  (NSCCA)  to  ensure  system  nucle¬ 
ar  safety  integrity.  Critical  functions 
are  those  which  apply  directly  to,  or 
control,  the  prearm,  arm,  fire,  unlock, 
release,  launch  or  targeting  functions 
of  a  nuclear  weapon  system.  The  con¬ 
tractor  should  be  instructed  to  discuss 
the  aspects  of  these  as  it  applies  to 
the  ATE  system. 
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4. 6. 2. 5  Software  Design  Approach.  Or¬ 
ganic  software  maintenance  considera¬ 
tions  may  dictate  that  HOL  be  used  wher¬ 
ever  reasonable  because  the  use  of  HOL 
normally  makes  that  maintenance  more 
efficient.  Also,  where  new  software  is 
to  be  built,  the  contractor  will  often 
be  directed  to  use  a  modular  design  ap¬ 
proach  such  as  the  top-down  structured 
approach.  General  statements  are  often 
used  to  call  for  software  design  which 
is  consistent,  logical,  and  well  docu¬ 
mented  in  accordance  with  stated  stan¬ 
dards. 

4. 6. 2. 6  Growth.  One  of  the  factors  to 
be  considered  in  ATE  design  is  the 
growth  capability  required  by  the  ATE 
computational  system.  Growth  potential 
must  accommodate  the  predicted  level  of 
computer  program  change  and  growth  activ¬ 
ity  over  the  anticipated  life  of  the  sys¬ 
tem.  The  contractor  should  be  directed 
to  estimate  the  required  growth  features 
such  as  spare  central  processing  time, 
spare  memory,  and  spare  input/output 
channel  capacity  and  provide  a  specified 
quantity  of  growth  capability. 

Following  the  preparation  and  submittal 
of  the  CCP,  the  Air  Force  ATE  software 
manager/engineer  will  participate  in  the 
review  and  approval  cycle.  Following 
approval  of  the  CCP,  work  can  begin  on 
the  process  of  computer  program  require¬ 
ment  specification. 

4.6.3  Control  and  Support  Software 
Requirements 

Much  of  the  control  and  support  software 
required  for  an  automatic  test  station 
can  be  purchased  from  the  ATE  computer 
manufacturer.  Many  times,  the  control 
and  support  software  requirements  are 
"defined"  by  studying  the  off-the-shelf 
programs  in  a  particular  computer  manu¬ 
facturer's  inventory  and  specifying  what 
is  available.  Some  systems  will  work 
acceptably  in  the  manner,  thus  this  tech¬ 
nique  can  be  used  to  some  extent.  A  more 
acceptable  approach  is  to  define  the 
requirements  for  the  specific  test  sys¬ 
tem  being  designed,  then  include  these 
requirements  in  an  RFP  to  prospective 


test  station  subcontractors.  This  ap¬ 
proach  may  require  the  subcontractor  or 
the  prime  contractor  to  develop  some  com¬ 
puter  programs  to  satisfy  this  require¬ 
ment.  These  requirements  for  control  and 
support  software  are  derived  by  the  weap¬ 
on  system  contractor  and  recorded  in  the 
prime  item  specification  to  be  included 
In  the  RFP  for  the  prospective  test  sta¬ 
tion  contractors. 

4. 6. 3.1  Control  Software  Requirements. 
The  purpose  of  ATE  control  software  Is 
to  provide  a  workable  test  system, 
providing  interface  between  the  test 

operator,  test  software  and  the  test 

equipment  including  the  ATE  computer. 

Consequently,  ATE  control  software 
requirements  cannot  be  derived  indepen¬ 
dently  from  the  test  equipment  and  in 
fact  must  be  derived  in  parallel  with 

the  equipment.  Following  the  LSA  and 
approval  of  the  SERDs  the  efforts 
required  for  specification  of  control 
software  requirement  are  as  follows: 

First,  a  refinement  of  the  general  con¬ 
cept  of  the  ATE  station,  used  in  devel¬ 
oping  SERDs,  must  be  accomplished.  Sec¬ 
ond,  in  parallel  but  slightly  lagging 
SERD  development,  a  study  must  be  accom¬ 
plished  to  document  the  physical  and 
functional  interfaces  of  all  UUTs  to  be 
tested.  Third,  an  estimate  must  then  be 
made  of  the  number  of  stimultaneous  stim¬ 
uli  and  sensor  measurements  which  are 
required  to  be  applied  by  each  stimuli 
for  each  UUT  via  the  physical  inter¬ 
faces.  The  total  set  of  UUTs  are  then 
examined  and  the  worst  case  of  simulta¬ 
neous  usage  for  each  stimuli  and  sensor 
is  determined.  The  overall  workload  of 
the  test  station  is  examined  to  deter¬ 
mine  the  total  number  of  UUTs  to  be 
tested  simultaneously.  Total  station 
through-put  will  impact  the  control  soft¬ 
ware  requirement. 

In  parallel  with  this  effort,  the  capa¬ 
bility  of  available  ATE  station  compu¬ 
ters  should  be  surveyed  with  emphasis  on 
capacity  and  speed.  Speed  is  essential 
to  provide  the  ATE  control  software  with 
adequate  timing  characteristics.  (The 
timing  requirement  generally  requires 
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that  much  of  the  executive  software  be 
written  in  assembly  language,  though  the 
rest  of  the  control  software  might  be 
written  In  an  HOL).  Computer  speed  is 
the  deciding  factor  in  determining 
whether  or  not  such  analogue  functions 
as  rise  times,  delay  times,  sequenced 
switching  times  can  or  should  be  accom¬ 
plished  by  hardware  or  ATE  control  soft¬ 
ware. 

At  this  point,  studies  should  be  con¬ 
ducted  jointly  by  the  UUT  and  ATE  engin¬ 
eers  trading  off  percent  of  achievable 
UUT  maintainability  versus  ATE  Station 
requirements  (both  hardware  and  soft¬ 
ware).  The  greater  the  required  achiev¬ 
able  UUT  maintainability  (over  and  above 
mandatory  end-to-end  testing)  the  more 
stringent  the  requirements  on  ATE  con¬ 
trol  software  in  all  three  areas  of  exec¬ 
utive  software  responsibility  (see  para¬ 
graph  4.1.1).  In  addition,  the  specifi¬ 
cation  of  other  modules  of  ATE  control 
software  are  affected  either  by  a 
greater  quantity  of  requirements  (e.g. , 
large  amount  of  test  equipment  to  be 
driven  and  more  interrupt  processes)  or 
by  a  requirement  to  handle  a  larger  vol¬ 
ume  of  data.  ATE  station  test  software 
requirements  are  correspondingly  greater 
and  the  quantity  of  UUT  test  software 
requirements  result  in  a  larger  volume 
of  ATLAS  statements.  The  eventual  result 
of  these  study  efforts  provides  a  set  of 
fundamental  (top  level)  requirements  for 
ATE  control  software. 

Prior  to  or  in  parallel  with  the  above 
studies,  the  ATE  control  software  must 
be  conceptually  configured  to  include 
the  functions  described  in  paragraph 
4.1.1,  ATE  control  software  definition. 
The  above  determined  top  level  require¬ 
ments  for  simultaneous  measurements  are 
analyzed  and  allocated  as  requirements 
on  the  ATE  control  software,  particu¬ 
larly  on  the  test  equipment  driver  and 
the  operating  system  software  functions. 

As  a  result  of  certain  LSA  efforts, 
(paragraph  4.4)  the  need  for  displaying 
and  providing  hard  copy  data  will  have 
been  established.  From  follow-on  LSA 
human  engineering  studies  the  form. 


content  and  timing  of  the  displayed  data 
and  hard  copy  data  will  have  been  deter¬ 
mined.  Both  of  these  study  results  pro¬ 
vide  specific  data  processing  require¬ 
ments  for  the  ATE  control  software, 
particularly  the  operating  system  and 
peripheral  driver  software. 

The  requirements  for  the  remaining  ATE 
control  software  modules  can  usually  be 
selected  from  available  specifications 
for  commercial  computers. 

4. 6. 3. 2  Support  Software  Requirements. 
ATE  support  software,  consists  of  the 
three  primary  classifications  discussed 
in  paragraph  4.1.2.  Computer  program 
requirements  are  derived  largely  from 
high  level  requirements  such  as  the  use 
of  FORTRAN  and  ATLAS,  and  the  necessity 
for  using  an  assembly  language.  Program 
development  aids  are,  for  the  most  part, 
standard  equipment  for  the  computer  manu¬ 
facturer.  ATE  computing  equipment  manu¬ 
facturers  have  developed  a  variety  of 
these  aids,  some  more  advanced  and 
sophisticated  than  others  and  some  with 
better  track  records  for  dependability. 
Figure  4.2.2  should  be  used  as  a  guide 
for  specifying  the  types  of  support  soft¬ 
ware  required.  In  many  cases  the  weapon 
system  contractor  has  his  own  library  of 
support  software  or  will  specify  support 
system  requirements  that  require  a  devel¬ 
opment  effort.  The  requirements  for  sta¬ 
tion  and  program  aids  result  from  cost 
versus  utility  studies.  Once  such'  aids 
are  determined  to  be  beneficial,  these 
requirements  enumerate  functions  and 
specific  inputs/outputs. 

There  are  occasions  when  it  is  desirable 
to  develop  ATE  computer  programs  on  a 
larger  more  powerful  computer  than  the 
one  selected  for  ATE.  This  Is  the  sub¬ 
ject  of  a  trade-off.  The  off-line  com¬ 
puter  represents  an  additional  expendi¬ 
ture  for  equipment  that  can  be  traded 
with  improvement  in  flow  times  for  new 
software  development  and  for  computer 
program  maintenance.  The  Improved  flow 
time  stems  from  a  larger  more  powerful 
computer  and  better  program  development 
aids.  Another  factor  is  the  avallabllty 
of  the  ATE  computer  if  testing  activity 
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is  high  and  whether  all  program  mainte¬ 
nance  activities  can  be  performed  as  a 
central  location.  The  choice  of  a  "host" 
computer  can  affect  the  requirement  for 
support  software,  e.g.,  this  type  of 
operation  would  require  a  cross  com¬ 
piler.  There  must  be  a  close  relation¬ 
ship  between  the  host  and  target  com¬ 
puter  hosted  support  software. 

In  summary  the  support  software  require¬ 
ments  specification  process  is  usually 
an  activity  performed  by  the  contractor 
using  his  experience,  expertise,  and 
knowledge  of  available  support  software. 
The  role  of  the  Air  Force  ATE  software 
manager/engineer  is  mainly  to  monitor 
the  process  and  to  give  advice  and  coun¬ 
cil  during  the  process.  Depending  on  the 
CCP  SOW,  the  Air  Force  may  or  may  not 
have  approval  rights  over  the  prime  item 
development  specifications  for  the  test 
station  which  includes  the  control  and 
support  software  requirement. 

4.6.4  Test  Station  Subcontract 

The  usual  method  for  a  weapon  system  con¬ 
tractor  to  acquire  an  automatic  test  set 
is  to  purchase  the  test  set  and  the  con¬ 
trol  and  support  software  from  a  sup¬ 
plier.  The  supplier  then  becomes  a  sub¬ 
contractor  to  the  weapon  systems  contrac¬ 
tor.  This  removes  the  Air  Force  ATE 
software  manager/engineer  further  from 
the  requirements  specification  process. 
The  Air  Force  has  no  official  jurisdic¬ 
tion  over  the  subcontractor,  only  the 
prime  contractor.  The  Air  Force  engineer 
is  usually  invited  to  attend  technical 
reviews  and  is  usually  a  recipient  of 
all  documentation  produced  by  the  subcon¬ 
tractor.  He  may  provide  counsel  and 
offer  suggestions  but  cannot  provide  di¬ 
rection  except  through  the  prime  contrac¬ 
tor.  Any  action  that  may  be  taken  that 
is  out  of  the  subcontract  must  be  nego¬ 
tiated  with  the  subcontractor  and  the 
prime  contractor  and  will  probably 
result  in  a  cost  adjustment. 

4.6.4. 1  RFP  Preparation.  Since  the  Air 
Force  has  no  jurisdiction  over  the  sub¬ 
contractor,  It  is  Important  for  the  soft¬ 
ware  manager/engineer  to  closely  monitor 


the  preparation  of  the  RFP  to  be  Issued 
by  the  prime  contractor  to  the  test  sta¬ 
tion  bidders. 

The  control  and  support  software  require¬ 
ments  described  In  the  previous  section 
're  recorded  in  a  Prime  Item  Development 
S  ecification  for  the  test  station.  This 
specification  is  usually  written  In 
accordance  with  MIL-STD-490.  If  the  test 
set  vendor  is  to  be  selected  through  com¬ 
petitive  bidding,  the  requirements  will 
be  general  and  will  not  address  a  partic¬ 
ular  vendor's  software  Implementation. 
The  specification  will  describe  what 
functions  the  software  must  provide  and 
any  implementation  requirements  which 
are  important  to  software  adequacy.  This 
specification  defines  each  of  the  soft¬ 
ware  functional  capabilities,  defines 
how  they  relate  to  one  another,  how  It 
ties  into  the  ATE  hardware  (computer, 
peripherals,  test  hardware)  and  defines 
the  qualification  test  and  acceptance 
test  requirements  for  the  software. 

In  addition,  the  RFP  should  Instruct  the 
test  station  subcontractor  to  prepare  a 
computer  program  development  specifica¬ 
tion  (as  specified  in  M1L-STD-483)  for 
control  or  support  software  that  may  be 
developed.  These  CPCI  development  spec¬ 
ifications  are  the  final  step  In  require¬ 
ment  specifications  for  control  and 
support  software.  The  RFP  should  also 
instruct  the  bidders  to  provide  program 
listings  and  other  design  description 
data  equivalent  to  a  CPCI  product  speci¬ 
fications  (also  specified  In  MIl-STD-483) 
for  off-the-shelf  computer  programs  that 
he  will  supply.  (It  should  be  noted  here 
that  any  new  programs  or  modifications 
to  existing  programs  that  may  be  devel¬ 
oped  by  the  prime  contractor  should  also 
require  a  CPCI  development  specifica¬ 
tion.) 

4. 6. 4. 2  Guidelines  for  Authorizing  and 
Monitoring  ATE  Control  and  Support  Soft¬ 
ware  Specification.  Depending  on  the  lan¬ 
guage  of  the  CCP  SOW,  the  ATE  software 
manager/engineer  may  have  the  authority 
for  review  and  approval  of  the  test  sta¬ 
tion  prime  item  development  specifica¬ 
tion  and  to  assist  in  the  evaluation  and 


award  of  a  contract  to  the  test  station 
subcontractor.  Assuming  this  is  the 
case,  the  appropriate  points  for  moni¬ 
toring  the  specification  of  control  and 
support  software  are  related  to  the 
usual  major  system  procurement  and  sys¬ 
tem  development  steps.  In  summary  the 
ATE  software  manager/engineer  must  be 
involved  in  the  following  activities: 

a.  Determining  if  the  specific  re¬ 
quirements  of  ATE  software  are  included 
in  the  ATE  procurement  package,  as 
described  in  paragraph  4.6.3; 

b.  Approving  the  statements  of  soft¬ 
ware  operability  and  software  require¬ 
ments  in  the  ATE  "A"  type  specification 
or  in  the  ATE  prime  item  development 
specification; 

c.  Assisting  in  ATE  contractor  eval¬ 
uation  by  assessing  the  ATE  contractor's 
system  software  development  credibility 
and  capability; 

d.  Determining  if  the  ATE  contractor 
is  compliant  with  the  intent  of  the  SOW 
and  CDRL  requirements  for  tasks  and  docu¬ 
mentation  of  software  requirements; 

e.  Determining  if  specific  attention 
has  been  given  to  the  maintenance  of  con¬ 
trol  and  support  software;  and 

f.  approving  the  development  specifi¬ 
cations. 

Paragraphs  4.8  through  4.11  of  AFR 
800-14,  which  cover  program  technical 
control  and  review,  are  applicable  to 
this  part  of  the  specification  of  ATE 
control  and  support  software.  If  ATE 
procurement  is  being  managed  by  a  con¬ 
tractor  for  ASD,  then  these  engineering 
management  requirements  should  be  made 
his  obligation. 

4. 6. 4. 3  Computer  Program  Development 
Specifications.  Computer  program  devel¬ 
opment  specifications  generated  by  the 
test  station  subcontractor  will  complete 
the  requirements  specification  process 
for  control  and  support  software.  These 
CPCI  development  specifications  contain 


the  detailed  requirements  for  computer 
programs  that  must  be  developed.  These 
specifications  should  be  reviewed  and 
approved  by  the  prime  contractor  prior 
to  the  software  Preliminary  Design 
Review  (PDR).  Formal  reviews  should  be 
conducted  to  evaluate  and  approve  the 
specifications.  The  Air  Force  is  nor¬ 
mally  invited  to  attend,  although  they 
have  no  official  jurisdiction  as  stated 
earlier.  Air  Force  representatives  may 
comment  and  provide  guidance  and  may 
offer  direction  only  through  the  prime 
contractor. 

4.7  TEST  SOFTWARE  REQUIREMENTS 

As  defined  previously,  test  software  con¬ 
sists  of  UUT  test  software,  ITA  test 
software  and  ATE  station  test  software. 
Test  software  requirements  depend  on  the 
selected  ATE  and  on  UUT  design  data, 
whether  the  UUT  be  the  ATE  station,  an 
ITA  or  a  Line  Replaceable  Unit  (LRU), 
and  on  the  defined  set  of  sequenced 
tests  which  have  been  approved  by  the 
UUT  design  organization.  Test  software 
requirements  specification  is  complete 
when  the  test  sequence  stimulus,  measure¬ 
ment  and  ancilliary  data  has  been  ap¬ 
proved,  or  an  approved  source  (such  as 
an  approved  TRD)  has  been  referenced. 

The  following  paragraphs  define  source 
data  for  each  category  of  test  software 
and  discuss  the  relationship  of  source 
data  to  TRDs  in  test  software  require¬ 
ments  development,  the  test  software 
requirements  development  process,  and 
the  test  software  development  specifica¬ 
tion.  Guidelines  for  the  involvement  of 
the  ATE  software  manager/engineer  are 
also  included  throughout. 

4.7.1  Test  Software  Requirements 
Source  Data 

4. 7. 1.1  LRU,  Secondary  Replaceable 
Unit  (SRU)  as  the  UUT.  Input  data  to  UUT 
test  software  requirements  (both  end-to- 
end  and  diagnostic  testing)  are: 

a.  UUT  level  and  UUT  sublevel  accep¬ 
tance  test  procedures  and  associated 
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test  requirements  (including  Contract 
End  Item  (CEI)  Specification); 

b.  UUT  level  and  UUT  sublevel  logic, 
functional  block,  schematic,  single  func¬ 
tion  and  wiring  diagrams; 

c.  Appropriate  references  to  replace¬ 
able  units  of  the  UUT; 

d.  Descriptions  from  a  qualified  de¬ 
sign  engineer  of  the  workings  of  the  UUT 
circuits  and  methods  of  troubleshooting 
them;  i.e. ,  diagnostic  testing  and  fault 
isolation.  The  purpose  of  these  data  are 
to  educate  the  contractor's  UUT  test 
software  designer  on  how  the  circuits 
work,  the  critical  functions  and  poten¬ 
tial  failure  points  so  that  he  can  prop¬ 
erly  select,  organize  and  sequence  the 
ATLAS  programming  effort  during  CPC  I 
development;  and 

e.  ATPG  data  for  UUT  digital  circuit 
functions. 

4. 7. 1.2  ITA  as  the  UUT.  The  source 
data  for  ITA  self-test  and  diagnostic 
software  and  test  software  for  adapters 
is  equivalent  to  that  described  in  the 
paragraph  above  with  the  letters  ITA 
substituted  for  UUT.  Subparagraph  (e) 
applies  only  to  programmable,  active 
adapters. 

4. 7. 1.3  ATE  Station  as  the  UUT.  The 
basic  requirements  for  ATE  station  test 
software  are  (1)  to  provide  a  means  for 
assuring  adequate  operability  of  the  ATE 
station  (usually  accomplished  by  end-to- 
end  tests),  and  (2)  to  provide  ATE  diag¬ 
nostic  test  capability.  The  source  data 
to  satisfy  both  basic  requirements  for 
ATE  station  test  software  is  equivalent 
to  that  described  in  the  paragraphs 
above.  An  ATE  station  acceptance  test 
fixture,  which  allows  more  complete  ATE 
end-to-end  testing  is  frequently  em¬ 
ployed.  Product  specifications  for  the 
ATE  are  the  source  of  both  performance 
and  diagnostic  test  sequences.  ATE  sta¬ 
tion  test  software  will  usually  be  pro¬ 
vided  by  the  ATE  contractor,  but  there 
are  examples  of  test  sets  that  are  both 
ATE  and  flight  readiness  equipment.  In 


the  latter  Instance,  station  tes 
sequences  nqy  be  prepared  by  the  "prime 
contractor  with  assistance  from  the  ATI 
manufacturer. 


Contractors  normally  separate  logistics 
engineering  from  design  engineering  orga¬ 
nizationally.  TRDs  are  prepared  under 
the  cognizance  of  the  contractor's  logis¬ 
tic  organization  for  the  AFLC  because 
logistics  personnel  deal  with  AFLC  con¬ 
stantly,  know  their  needs,  problems,  and 
how  they  operate. 

Historically,  the  contractor's  logistics 
organization  has  prepared  TSs  for  end- 
to-end  and  diagnostic  testing  of  UUTs  in 
parallel  with  software  development  and 
then  demonstrated  their  correctness  by 
technical  order  verification  and  valida¬ 
tion  (V&V).  Technical  Order  (T.O.)  VSV 
has  been  conducted,  as  a  "hands  on"  op¬ 
eration  for  participating  AF  personnel. 

The  contractor's  logistic  organization 
obtained  the  T.O.  source  data  by  request¬ 
ing  T.O.  inputs  from  design  engineering. 
These  input  requests  were  processed 
through  the  contractor's  change  board, 
then  scheduled  and  documented  as  formal 
data  packages  to  be  provided  by  design 
engineering  to  the  logistics  organiza¬ 
tion. 


Avionics  design  engineers  developing  the 
UUT  and  ITA  test  software  will  normally 
be  located  in  the  contractor's  design 
engineering  organization.  The  major 
problem  associated  with  test  software 
development  is  not  "were  the  tests 
written  correctly  in  ATLAS?,"  rather 


4.7.2  Relationship  of  Test  Software 
Source  Data  and  Test  Require¬ 
ments  Document  (TRD) 


Currently  these  T.O.  input  data  packages 
are  to  be  included  as  part  of  deliver¬ 
able  TRDs  (in  addition  to  the  T.O.s). 
This  will  not,  however,  change  the  con¬ 
tractor's  internal  mechanism  for  devel¬ 
oping  the  data  packages.  Design  engi¬ 
neering  will  continue  to  develop  these 
inputs  and  forward  them  to  the  logistics 
organization  for  incorporation  into  TRDs. 
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"were  the  correct  tests  written  In 
ATLAS?"  This  problem  can  only  be  solved 
by  comprehensive  review  by  the  design 
engineer  and  the  test  software  engineer, 
using  either  ATLAS  language  or  English 
language  statements,  depending  on  which 
provides  the  best  means  of  communication. 

The  contractor's  UUT  and  ITA  test  soft¬ 
ware  designers  will  obtain  their  source 
data  directly  from  TRD  data  provided  by 
design  engineering  and -not  wait  for  the 
logistics  organization  to  prepare  and 
process  TROs.  In  some  instances  these 
source  data  are  provided  in  the  ATLAS 
language. 

TRDs  are  only  one  source  of  data  for  UUT 
and  ITA  test  software  definitive  require¬ 
ments.  Other  sources  are  the  many  infor¬ 
mal  discussions  with  UUT  design  engi¬ 
neers,  where  specific  questions  are 
answered,  circuit  understanding  is  ob¬ 
tained,  and  design  features  are  inter¬ 
preted. 

The  AF  ATE  software  manager/engineer  has 
the  opportunity  to  review  these  test 
sequence  data  at  formally  scheduled 
TRDs.  These  data  are  scheduled  in  a 
sequential  series  starting  with  require¬ 
ments  for  end-to-end  testing  and  continu¬ 
ing  with  sets  of  diagnostic  testing 
requirements. 

4.7.3  UUT  Test  Software  Requirements 
Development 

The  end-to-end  test  concept  and  general 
plan  can  be  started  immediately  after 
the  UUT  production  acceptance  test  proce¬ 
dure  is  available.  The  diagnostic  test 
concept  and  general  plan  is  delayed  due 
to  the  dependence  on  TRD  input  releases. 

4. 7. 3.1  End-to-End  Tests.  The  UUT  end- 
to-end  software  requirements  definition 
process  is  illustrated  by  Figure  4.7-1. 
Available  UUT  source  data  (see  paragraph 
4.7.1)  for  end-to-end  tests  are  compiled 
and  put  into  a  form  facilitating  the  gen¬ 
eration  of  UUT  test  concept  and  general 
plan.  The  total  functional  capability  of 
a  UUT  is  broken  down  into  a  set  of  logi¬ 
cal  subfunctions,  the  proper  operation 


of  which  will  be  verified  by  a  set  of 
tests;  e.g. ,  synchro  operation  under 
varying  power  conditions.  The  number  of 
UUT  test  points  impacts  the  programming 
test  and  software  development  cost.  This 
leads  to  a  trade-off  between  availabil¬ 
ity  of  test  point  and  program  complexity 
with  cost  as  the  primary  criterion. 
Depending  upon  the  complexity  of  the  UUT 
it  may  be  desirable  to  make  one  or  more 
additional  levels  of  functional  break¬ 
down.  A  UUT  general  test  plan  is  gener¬ 
ated  which  defines  the  selected  sequence 
of  subfunctions. 

Upon  completion  of  the  UUT  test  concept 
and  general  plan,  an  in-house  review  is 
held  with  the  UUT  design  engineer(s),  to 
(1)  assure  that  the  correct  UUT  design 
and  test  requirements  data  were  used, 
and  (2)  to  establish  approval  of  the  UUT 
test  concept  and  general  plan. 

The  contractor's  UUT  test  software  engi¬ 
neer  then  lists  all  the  detailed  tests 
he  intends  to  perform  to  exercise  ade¬ 
quately  each  UUT  function  in  a  manner 
which  will  satisfy  the  UUT  acceptance 
test  procedure.  These  tests  are  then 
sequenced  and  flow  diagrams  may  be  pre¬ 
pared.  Figure  4.7-2  is  an  example  of  a 
simple  diagnostic  flow  diagram  for  cer¬ 
tain  synchro  operations.  The  main  flow 
portion  of  the  diagram  could  be  a  part 
of  an  end-to-end  acceptance  test.  The 
flow  diagrams  are  then  reviewed  with  the 
UUT  design  engineer  for  concurrence  that 
these  tests  will  satisfy  the  UUT  accep¬ 
tance  test  procedure  or  equivalent.  At 
the  present  time,  english  language  state¬ 
ments  are  preferable  because  many  design 
engineers  do  not  have  working  knowledge 
of  ATLAS,  although  with  some  training 
this  problem  can  be  overcome,  and  re¬ 
views  are  simplified.  With  the  current 
level  of  familiarity  with  ATLAS,  English 
language  flow  diagrams  have  particular 
value  as  a  tool  assuring  that  changes  in 
ATE  stations  or  UUT  configurations  are 
adequately  incorporated.  As  the  ATLAS 
language  becomes  better  known,  it  may 
provide  a  more  precise  method  of  com¬ 
munication  than  English. 
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The  form  in  which  the  UUT  test  software 
requirements  is  presented  is  the  subject 
of  a  trade-off.  There  are  two  opposing 
views.  One  is  that  TRD  data  should  be 
written  in  English  by  the  UUT  design  en¬ 
gineer  and  converted  to  ATLAS  by  a  test 
software  engineer.  The  second  is  that 
the  TRD  data  should  be  written  directly 
in  ATLAS  by  the  UUT  design  engineer. 
Arguments  for  the  first  case  are  alluded 
to  in  the  above  discussion.  The  argument 
for  the  second  case  is  that  there  may  be 
an  information  loss  in  translating  the 
UUT  designer  inputs  to  the  flow  charts 
and  into  ATLAS  and  that  a  review  cycle 
could  be  eliminated.  It  is  currently 
true  that  most  design  engineers  are  not 
familiar  with  ATLAS,  but  as  stated  pre¬ 
viously  this  problem  can  be  overcome 
with  adequate  training.  If  the  UUT  test 
sequences  are  preferred  in  ATLAS,  the 
test  software  engineer  should  provide 
appropriate  guidelines  for  the  UUT 
design  engineer  to  follow  when  writing 
ATLAS  statements. 

Certain  groupings  of  these  test  proce¬ 
dures  will  be  identified  as  CPCIs.  For 
each  CPCI  certain  configuration  manage¬ 
ment  functions  must  be  performed  such  as 
preparation  of  development  specifica¬ 
tions  and  product  specifications  and 
holding  preliminary  and  critical  design 
reviews  (PDRs  and  CDRs).  The  design 
reviews  provide  an  opportunity  for  the 
Air  Force  to  participate  in  the  review 
process  and  to  offer  guidance  and  direc¬ 
tion  as  necessary.  The  test  software 
development  specification  is  described 
in  paragraph  4.7.6.  The  test  sequence 
flow  charts  and  the  ATLAS  statements  are 
included  in  the  product  specification 
and  are  reviewed  at  the  CDR  for  the 
specific  CPCI. 

4. 7. 3. 2  Diagnostic  Tests.  The  process 
for  development  of  UUT  diagnostic  test 
software  requirements  follows  approxi¬ 
mately  the  same  plan  as  that  for  UUT 
end-to-end  test  software.  However,  the 
plan  is  cycled  only  once  for  the  end-to- 
end  requirements;  but  for  diagnostics, 
the  plan  is  repeated  a  number  of  times 
(depending  upon  the  complexity  of  the 
UUT).  Diagnostic  requirements  are  speci¬ 


fied  for  each  set  of  faults.  Also,  con¬ 
siderable  circuit  analysis  must  be  accom¬ 
plished  to  determine  and  understand  what 
failures  could  cause  similar  or  identi¬ 
cal  UUT  malfunctions  and  to  generate 
unique  diagnostic  test  flow  diagrams. 

4.7.4  ATE  Station  Test  Software 
Requirements  Development 

ATE  station  test  software  requiremants 
are  concerned  with  ATE  station  self-test 
as  defined  in  paragraph  4.1.3  under  test 
station  test  software.  The  requirements 
are  developed  in  precisely  the  same  man¬ 
ner  as  for  UUT  test  software  require¬ 
ments  development  (paragraph  4.7.3). 
Source  data  is  defined  in  paragraph 
4. 7. 1.3. 

4.7.5  ITA  Test  Software  Requirements 
Development 

ITA  test  software  requirements  are  con¬ 
cerned  with  ITA  self-test.  The  require¬ 
ments  are  developed  by  the  same  process 
as  UUT  test  software  requirements  devel¬ 
opment  (paragraph  4.7.3).  Source  data  is 
defined  in  paragraph  4. 7. 1.2. 

4.7.6  Test  Software  Development 
Specification 

Test  software  requirements  fall  into  two 
categories.  These  are  UUT-dependent  re¬ 
quirements;  e.g. ,  UUT  test  sequences, 
stimulus  and  measurement  requirements; 
and  ATE-dependent  requirements  relating 
to  the  ATE  environment.  Once  the  ATE  has 
been  selected,  software  requirements 
(principally  functional  and  physical  in¬ 
terfaces)  are  imposed  on  the  UUT  CPCIs. 
A  development  specification  must  contain 
the  ATE-dependent  requirements,  -UUT 
dependent  test  sequences  and  quality 
assurance  provisions.  The  ATE-dep%Bent 
and  UUT-dependent  requirements  are  dis¬ 
cussed  separately  in  the  following  two 
paragraphs. 

4.7.6. 1  ATE-Environment  Requirements. 
These  requirements  are  unique  to  a  CPCI, 
even  excluding  the  detailed  UUT  test 
sequences,  stimulus  and  measurement 
requirements.  ATE  environment  requi re- 
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W"  if 


ments  definition  can  begin  and  a  prelim¬ 
inary  CPCI  development  specification 
prepared  when: 

a.  ATE  selection  and  procurement  has 
progressed  so  that  functional  and  physi¬ 
cal  interface  requirements  and  con¬ 
straints  on  UUT  CPCIs  can  be  estab¬ 
lished;  and 

b.  UUT  designs  have  progressed  so 
that  the  requirements  imposed  on  the 
performance,  design,  test,  and  qualifi¬ 
cation  of  the  UUT  test  software  can  be 
established. 

The  details  of  the  UUT  performance  and 
diagnostic  tests  themselves  will  not 
have  been  completed,  but  the  functional 
areas  will  have  been  defined.  For  exam¬ 
ple,  it  will  be  possible  to  make  state¬ 
ment  such  as: 

"This  Test  Software  (CPCI)  shall  be 
subdivided  into  the  following  three 
functional  areas: 

a.  Mode  Control 

b.  Protection  Circuit  Tests 

c.  Requlator  Output  Tests." 

The  CPCI  Development  Specification  may 
be  preliminary,  and  should  contain: 

UUT  Test  Computer  Program  (CPCI) 
Definition 

Interface  requirements  and  descrip¬ 
tion 

Physical  interfaces  UUT  to 
ATE/ITA 

Functional  interfaces,  e.g. , 
Electrical 

Control  Software  (if  required) 

Functional  Requirements  and 

Description 

UUT  Test  identified  and 

described  functionally,  e.g.. 

Inputs 

Processing 

Outputs 


Quality  Assurance  Requirements 
Type  and  extent  of  verification 
requi red 
Inspection 
Analysis 
Test 

Demonstration 
Test  Environment 
Test  Requirements,  e.g. 
Correlation  of  type  of  test  to 
section  3  requirements 

As  can  be  seen,  a  general  understanding 
of  the  UUT  functions  and  specific  under¬ 
standing  of  ATE  capabilities  is  required 
to  prepare  the  development  specification 
as  described  above.  This  allows  effec¬ 
tive  management  of  not  only  the  require¬ 
ments  specification  process,  but  also 
all  of  the  subsequent  development. 

4. 7.6.2  UUT  Dependent  Requirements. 
Test  software  CPCI  development  specifi¬ 
cations  must,  as  a  minimum,  identify  the 
UUT  tests  to  be  performed  and  reference 
the  appropriate  TRD  data  that  will  con¬ 
tain  the  detailed  test  sequences,  stim¬ 
ulus  and  responses.  Since  the  TRD  data, 
particularly  diagnostic  tests,  is 
usually  produced  well  downstream  from 
the  CPCI  development  specification  these 
data  are  not  available  when  the  develop¬ 
ment  specification  is  generated.  The  ATE 
environment  requirements  are  available 
early  and  with  the  identification  of  the 
test  sequences  and  appropriate  refer¬ 
ences  to  TRD  data,  should  be  sufficient 
for  the  development  specification. 

If  the  TRD  data  are  to  be  written  direct¬ 
ly  in  ATLAS,  the  development  specifica¬ 
tion  should  include  specification 
requirements  for  writing  the  ATLAS  state¬ 
ments  for  the  purpose  of  ensuring  the 
resulting  source  code  is  compatible  with 
the  rest  of  the  test  software  CPCI. 
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4.8  ATE  SOFTWARE  REQUIREMENTS  SUMMARY 

The  ATE  requirements  specification  pro¬ 
cess  begins  with  a  ROC  for  a  weapon 
system.  The  reference  here  may  only 
state  that  adequate  support  equipment  be 
provided  for  effective  maintenance.  The 
weapon  systems  RFP  may  contain  signifi¬ 
cant  ATE  requirements  particularly  if 
ATE  development  is  to  be  included  in  the 
prime  contract.  However,  the  real  begin¬ 
ning  is  the  definition  of  the  ATE  result¬ 
ing  from  a  LSA  and  documented  in  a  SERD. 
Following  approval  of  the  SERDs  and  the 
appropriate  contract  arrangements,  work 
may  begin  specifically  on  ATE  software 
requirements  specification.  ATE  software 
falls  into  three  general  classifica¬ 
tions:  Control  software,  support  soft¬ 
ware  and  test  software.  Test  software  is 
further  divided  into  UUT  test  software, 
station  test  software  and  ITA  test  soft¬ 
ware.  Control  software,  support  software 
and  station  test  software  are  all  depend¬ 
ent  on  the  ATE  and  its  environment.  The 
schedule  for  requirements  specifications 
for  these  types  of  software  is  similar 
and  occurs  prior  to  the  test  software. 
The  remaining  test  software;  i.e.,  UUT 
and  ITA  test  software,  are  dependent  on 
production  UUT's  and  the  requirements 
specification  process  must  of  necessity 


lag  behind  the  other  types.  Given  the 
various  sources  of  ATE  requirements  to 
specification  process  is  not  unlike  any 
other  software  requirements  specifica¬ 
tion  process,  requiring  in  the  final 
reckoning,  a  development  specification 
stating  the  required  interface,  func¬ 
tions  to  be  performed,  performance  re¬ 
quired  and  quality  assurance  provisions. 

The  ATE  software  requirements  specifica¬ 
tion  process  is  largely  a  contractor 
activity  that  must  be  monitored  closely 
by  the  Air  Force  ATE  software  manager/ 
engineer.  The  Air  Force  has  the  opportun¬ 
ity  to  influence  the  process  in  the  prep¬ 
aration  of  the  weapon  system  RFP,  guid¬ 
ance  and  consulting  during  the  LSA 
process,  approval  of  the  SERDs,  prepara¬ 
tion  of  the  SOW  for  the  contract  supple¬ 
ment  agreement,  guidance  and  consulting 
in  the  preparation  of  the  ATE  Prime  item 
development  specification  for  ATE  pro¬ 
curement  (possibly  approval,  if  so 
stated  in  the  contract  supplement)  and 
in  participating  in  the  schedule  PDRs 
and  CDR  for  the  various  CPCIs. 

Table  4.8-1  provides  a  checklist  of  sig¬ 
nificant  considerations  that  should  be 
made  in  the  requirements  specification 
process. 
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Table  4.8-1.  A  TE  Software  Requirements  Specification  Checklist 


1.  Has  the  contractual  method  for  acquiring  ATE  been  decided  and  identified: 
i.e. ,  inclusion  in  the  weapon  system  contract,  contract  supplement,  or 
separate  contract? 

2.  Does  the  weapon  system  contract  contain  provisions  for  a  logistics  support 
analysis  and  does  the  CDRL  require  SERD's  for  ATE? 

3.  Does  the  LSA  specifically  address  ATE? 

4.  Are  there  SERD's  that  define  the  ATE? 

5.  Does  the  SOW  for  a  contract  supplement  for  ATE  procurement  contain  definitive 
words  regarding  the  tasks  to  be  performed  and  the  quantity  and  quality  of  the 
expected  products? 

6.  Does  the  SOW/CDRL  for  the  contract  supplement  require  development  and  product 
specifications  for  each  ATE  CPCI?  Does  it  contain  provisions  for  all  necessary 
data  rights? 

7.  Has  sufficient  lead  time  been  scheduled  for  the  development  of  ATE  software 
requirements  with  respect  to  the  availability  of  the  production  UUT? 

8.  Has  a  prime  item  development  specification  been  prepared  for  ATE  suppliers  and 
does  it  include  requirements  for  ATE  control  and  support  software? 

9.  Does  the  SOW  for  the  ATE  supplier  require  a  development  specification  for  all 
newly  developed  software  and  a  product  specification  or  equivalent  data  for 
all  delivered  software?  Are  there  provisions  for  technical  reviews  such  as 
PDR's  and  CDR's? 

10.  Does  the  SOW  for  the  ATE  supplier  provide  for  all  necessary  data  rights? 

11.  Will  the  TRD  data  specifying  UUT  test  sequences  be  written  in  English  or 

ATLAS?  1 

12.  Have  all  PDR's  and  CDR's  been  attended? 

13.  Do  the  requirements  for  control  software  contain  the  provisions  specified  in 
Figure  4.1-1? 

14.  Do  the  requirements  for  support  software  contain  the  provisions  specified  in 
Figure  4.1-2? 

15.  Do  the  requirements  for  test  software  contain  the  provisions  specified  in 
Figure  4.1-3? 

16.  Do  the  test  software  CPCI  development  tests  contain  appropriate  data  as 
specified  in  paragraph  4. 7.6.1? 

17.  Have  total  workload  requirements  for  the  ATE  been  defined?  How  many  UUTs  must 
be  tested  simultaneously?  Has  the  worst  cast  situations  been  identified? 
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Section  6.0  MATRIX:  GUIDEBOOK  TOPICS  VS.  GOVERNMENT  DOCUMENTS 

The  elements  in  Figure  6.0-1  correspond 
to  the  sections  in  the  government,  publi¬ 
cation  wherein  the  corresponding  topic 
is  discussed  to  the  largest  extent. 
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Figure  6.0 1 1.  Guidebook  Topic*  Versus  Government  Documentation  (Sheet  1  of  2) 


Section  7.0  GLOSSARY  OF  TERMS 


Acquisition  Engineer  -  Military  or  civil¬ 
ian  member  of  a  SPO  or  an  AFSC  division 
who  supports  the  activities  of  a  SPO. 

Computer  Program  -  A  series  of  in- 
struct  ions  or  statements  in  a  form 
acceptable  to  computer  equipment, 
designed  to  cause  the  execution  of  an 
operation  or  series  of  operations.  Com¬ 
puter  programs,  and  maintenance/diagnos¬ 
tic  programs.  They  also  include  applica¬ 
tions  programs  such  as  payroll,  inven¬ 
tory,  control,  operational  flight,  stra¬ 
tegic,  tactical,  automatic  test,  crew 
simulator  and  engineering  analysis  pro¬ 
grams.  Computer  programs  may  be  either 
machine  dependent  or  machine  indepen¬ 
dent,  and  may  be  general  purpose  in 
nature  or  be  designed  to  satisfy  the 
requirements  of  a  specialized  process  of 
a  particular  user. 

Computer  Program  Comfiguration  Items  -  A 
computer  program  or  aggregate  of  related 
computer  programs  designated  for  config¬ 
uration  management.  A  CPCI  may  be  a 
punched  deck  of  cards,  paper  or  magnetic 
tape  or  other  media  containing  a  se¬ 
quence  of  instructions  and  data  in  a 
form  suitable  for  insertion  in  a  digital 
computer. 

Configuration  Item  -  An  aggregation 
which  satisfies  an  end  use  function  and 
is  designated  for  configuration  manage¬ 
ment. 

Configuration  Control  -  A  management  dis¬ 
cipline  applying  technical  and  adminis¬ 
trative  direction  and  surveillance  to: 

a.  Identify  and  document  the  func¬ 
tional  and  physical  characteristics  of  a 
configuration  item 

b.  Control  changes  to  those  character¬ 
istics;  and 

c.  Record  and  report  change  process¬ 
ing  and  implementation  status 


Control  Software  -  Software  used  during 
execution  of  a  test  program  which  con¬ 
trols  the  nontesting  operations  of  the 
ATE.  This  software  is  used  to  execute  a 
test  procedure  but  does  not  contain  aqy 
of  the  stimuli  or  measurement  parameters 
used  in  testing  a  unit  under  test.  Where 
test  software  and  control  software  are 
combined  in  one  inseparable  program, 
that  program  will  be  treated  as  test 
software  (AFLC  66-37). 

Data  Base  -  A  collection  of  program 
code,  tables,  constants,  interface  ele¬ 
ments  and  other  data  essential  to  the 
operation  of  a  computer  program  or  soft¬ 
ware  subsystem. 

External  Interface  -  Data  passed  between 
two  or  more  computer  programs  or  between 
a  computer  program  and  peripheral  de¬ 
vices  external  to  the  computer  in  which 
the  program  resides.  The  data  may  be  in 
the  form  of  an  interrupt  signal  or  may 
be  a  digital  data  stream  either  output 
from  the  computer  or  input  into  the  com¬ 
puter  for  processing. 

Instructional  System  -  That  portion  of  a 
TS  which  supports  the  instructor's  func¬ 
tions.  It  consists  of  hardware  and  soft¬ 
ware  used  by  the  instructor  to  communi¬ 
cate  with  trainees  to  control  the  state 
of  the  simulator  by  insertion  of  faults 
and  to  display  simulator  status  and  stu¬ 
dent  responses. 

Internal  Interfaces  -  Data  passed  be- 
tween  elements  of  a  computer  program  and 
usually  included  in  the  computer  program 
data  base. 

Logic  Flow  -  A  diagrammatic  representa¬ 
tion  of  the  logic  sequence  for  a  com¬ 
puter  program.  Logic  flows  may  take  the 
form  of  the  traditional  flow  charts  or 
in  some  other  form  such  as  a  program 
design  language. 
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Organic  -  A  term  used  to  designate  a 
task  performed  by  the  Air  Force  rather 
than  a  contractor. 

Product  Baseline  -  The  final  approved 
conf i gurat i on  i dent i f i cat i on.  It  identi¬ 
fies  the  as  designed  and  functionally 
tested  computer  program  configuration. 
It  is  defined  by  the  Computer  Program 
Product  Specification. 

Program  Design  Language  -  An  English- 
like,  specially  formatted,  textual  lan¬ 
guage  describing  the  control  structure, 
logic  structure,  and  general  organiza¬ 
tion  of  a  computer  program.  Essential 
features  of  a  program  design  language 
are: 

a.  It  is  an  English-like  representa¬ 
tion  of  a  computer  procedure  that  is 
easy  to  read  and  comprehend. 

b.  It  is  structured  in  the  sense  that 
it  utilizes  the  structured  programming 
control  structures  and  indentation  to 
show  nested  logic. 

c.  It  uses  full  words  or  phrases 
rather  than  the  graphic  symbols  used  in 
flow  charts  and  decision  tables. 

Quality  Assurance  -  A  planned  and  system¬ 
atic  pattern  of  all  software-related 
actions  necessary  to  provide  adequate 
confidence  that  computer  program  config¬ 
uration  items  or  products  conform  to 
establish  software  technical  require¬ 
ments  and  that  they  achieve  satisfactory 
performance. 

Software  -  A  combination  of  associated 
computer  programs  and  computer  data  re¬ 
quired  to  enable  the  computer  equipment 
to  perform  computational  or  control  func¬ 
tions. 

Software  Quality  -  The  primary  charac¬ 
teristic  of  software  quality  is  that  the 
software  performs  as  intended.  This  im¬ 
plies  not  only  that  the  software  re¬ 
flects  the  specification  to  which  it  is 
written  but  also  that  the  software  speci¬ 
fications  themselves  adequately  address 
the  system/mission  requirements.  Key 


attributes  of  software  quality  include: 
reliability,  flexibility,  traceability, 
testability,  integrity,  maintainability, 
and  completeness.  Quality  software  is: 
well-defined,  well -documented,  free  of 
design  deficiencies  and  coding  errors, 
satisfies  performance  requirements,  and 
has  minimum  life  cycle  cost. 

Source  Selection  -  The  process  of  select¬ 
ing  which  among  competing  contractors 
shall  be  awarded  a  contract.  A  signifi¬ 
cant  portion  of  this  involves  evaluation 
of  proposals  to  determine  the  degree  to 
which  the  government's  requirements 
would  be  satisfied. 

SPO  Cadre  -  Nucleus  of  a  SPO  formed  by 
an  AFSC  division  in  accordance  with  AFR 
800-2. 

Support  Software  -  Auxiliary  software 
used  to  aid  in  preparing,  analyzing  and 
maintaining  other  software.  Support  soft¬ 
ware  is  never  used  during  the  execution 
of  a  test  program  on  a  tester,  although 
it  may  be  resident  either  on-line  or 
off-line.  Included  are  assemblies,  com¬ 
pilers,  translators,  loaders,  design 
aids,  test  aids,  etc.  (AFLC  66-37). 

System  Engineering  -  The  application  of 
scientific  and  engineering  efforts  to 
transform  an  operational  need  or  state¬ 
ment  of  deficiency  into  a  description  of 
systems  requirements  and  a  preferred  sys¬ 
tem  configuration  that  has  been  opti¬ 
mized  from  a  life  cycle  viewpoint.  The 
process  has  three  principal  elements: 
functional  analysis,  synthesis,  and 
trade  studies  or  cost-effectiveness  opti¬ 
mization. 

Test  Software  -  Programs  which  implement 
documented  test  requirements.  There  is  a 
separate  test  program  written  for  each 
distinct  configuration  of  unit  under 
test  (AFLC  66-37). 

Top  Down  Structured  Programs  -  Struc¬ 
tured  programs  with  the  additional 
characteristics  of  the  source  code  being 
logically,  but  not  necessarily  physi¬ 
cally,  segmented  in  a  hierarchical  man¬ 
ner  and  only  dependent  on  code  already 


written.  Control  of  execution  between 
segments  is  restricted  to  transfer 
between  vertically  adjacent  hierarchical 
segments. 

Validation  -  Computer  program  validation 
is  the  test  and  evaluation  of  the  com¬ 
plete  computer  program  aimed  at  ensuring 
compliance  with  the  performance  and 
design  criteria. 

Verification  -  Computer  program  verifi¬ 
cation  is  the  iterative  process  of  con¬ 
tinuously  determining  whether  the  pro¬ 
duct  of  each  step  of  the  computer  pro¬ 
gram  acquisition  process  fulfills  all 
requirements  levied  by  the  previous 
step,  including  those  set  for  quality. 


System  Life  Cycle  -  The  system  acquisi¬ 
tion  life  cycle  consists  of  the  follow¬ 
ing  five  major  phases  with  major  deci¬ 
sion  points: 

a.  Conceptual  phase 

b.  Validation  phase 

c.  Full-scale  development  phase 

d.  Production  phase 

e.  Deployment  phase 
(AFR-800-14,  Volume  II) 
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ADI 

Attitude  Direction  Indicator 

DID 

Data  Item  Description 

AF 

Ai r  Force 

DRLMS 

Digital  Radar  Land  Mass 
Simulator 

AFLC 

Air  Force  Logistics  Command 

FEMA 

Failure  Modes  and  Effects 

AFSC 

Air  Force  Systems  Command 

Analysis 

AGERD 

Aerospace  Ground  Equipment 
Requirements  Documentation 

FORTRAN 

Formula  Translator 

HOL 

High  Order  Language 

AI 

Adapter  Interface 

HSI 

Horizontal  Situation  Indicator 

ASD 

Aeronautical  Systems  Division 

I/O 

Input/Output 

ATE 

Automatic  Test  Equipment 

IOC 

Initial  Operational  Capability 

ATLAS 

Abbreviated  Test  Language  for 

AI 1  Systems 

ISD 

Instructional  Systems 
Development 

ATPG 

Automatic  Test  Pattern 

Generator 

ITA 

Interface  Test  Adapter 

CCP 

Contract  Change  Proposal 

LCC 

Life  Cycle  Cost 

CDR 

Critical  Design  Review 

LRU 

Line  Replaceable  Unit 

CDRL 

Contract  Data  Requirements 

List 

LSA 

Logistics  Support  Analysis 

LSC 

Logistics  Support  Costs 

CEI 

Contract  End  Item 

MTBO 

Mean  Time  Between  Overhaul 

CPCI 

Computer  Program  Configuration 
Item 

NSCCA 

Nuclear  Safety  Cross-Check 
Analysis 

CPDP 

Computer  Program  Development 

Plan 

ORLA 

Optimum  Repair  Level  Analysis 

CPU 

Central  Processing  Unit 

PD 

Preliminary  Design 

CRISP 

Computer  Resources  Integrated 

PDR 

Preliminary  Design  Review 

Support  Plan 

PMD 

Program  Management  Directive 

CRT 

Cathode  Ray  Tube 

PMP 

Program  Management  Plan 

CRWG 

Computer  Resources  Working 

Group 

RDT&E 

Research  Design  Test  and 
Evaluation 

DCP 

Development  Concept  Paper 

RFI 

Radio  Frequency  Interference 

DDP 

Design  Data  Package 
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RFP 

Request  for  Proposal 

SRU 

Secondary  Replacement  Unit 

ROC 

Required  Operational 
Capabilities 

TI 

Technical  Interchange 

T.O. 

Technical  Order 

SAE 

Software  Acquisition 
Engineering 

TRD 

Test  Requirement  Document 

SERD 

Support  Equipment 

TS 

Trainer  Simulator 

Recommendations  Data 

UUT 

Unit  Under  Test 

SOW 

Statement  of  Work 

V&V 

Verification  and  Validation 

SPO 

System  Program  Office 
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